Skip to content

GitLab

Unverified Commit b4ac4fa0 authored by Lucia Fang's avatar Lucia Fang Committed by GitHub
Browse files

[model] make llama4 compatible with pure dense layers (#17315) 


Signed-off-by: default avatarLucia Fang <fanglu@fb.com>
parent e1360005
Show whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
vllm/model_executor/models/llama4.py
View file @ b4ac4fa0
...@@ -273,8 +273,8 @@ class Llama4DecoderLayer(nn.Module): ...@@ -273,8 +273,8 @@ class Llama4DecoderLayer(nn.Module):
cache_config=cache_config, cache_config=cache_config,
prefix=f"{prefix}.self_attn", prefix=f"{prefix}.self_attn",
) )
is_moe_layer = (self.layer_idx + is_moe_layer = config.interleave_moe_layer_step > 0 and (
1) % config.interleave_moe_layer_step == 0 self.layer_idx + 1) % config.interleave_moe_layer_step == 0
if is_moe_layer: if is_moe_layer:
self.feed_forward = Llama4MoE( self.feed_forward = Llama4MoE(
config=config, config=config,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment