Skip to content

GitLab

Unverified Commit 3d158cdc authored by wchen61's avatar wchen61 Committed by GitHub
Browse files

Add default value to avoid Falcon crash (#5363) (#10347) 


Signed-off-by: default avatarwchen61 <wchen61@foxmail.com>
parent 02dbf30e
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 0 deletions
vllm/model_executor/models/falcon.py
View file @ 3d158cdc
......@@ -250,6 +250,9 @@ class FalconDecoderLayer(nn.Module):
self.mlp = FalconMLP(config, quant_config)
self.config = config
if (not hasattr(config, "num_ln_in_parallel_attn")):
config.num_ln_in_parallel_attn = None
if (config.num_ln_in_parallel_attn is None
and config.new_decoder_architecture):
config.num_ln_in_parallel_attn = 2
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment