Skip to content

GitLab

Unverified Commit 5204ff5c authored by Cyrus Leung's avatar Cyrus Leung Committed by GitHub
Browse files

[Bugfix] Fix Granite 3.0 MoE model loading (#12446) 


Signed-off-by: default avatarDarkLight1337 <tlleungac@connect.ust.hk>
parent 0cc6b383
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
vllm/model_executor/models/granitemoe.py
View file @ 5204ff5c
...@@ -348,6 +348,7 @@ class GraniteMoeForCausalLM(nn.Module, SupportsLoRA, SupportsPP): ...@@ -348,6 +348,7 @@ class GraniteMoeForCausalLM(nn.Module, SupportsLoRA, SupportsPP):
self.config = config self.config = config
self.lora_config = lora_config self.lora_config = lora_config
self.quant_config = quant_config # Required by MixtralForCausalLM
self.model = GraniteMoeModel(vllm_config=vllm_config, self.model = GraniteMoeModel(vllm_config=vllm_config,
prefix=maybe_prefix(prefix, "model")) prefix=maybe_prefix(prefix, "model"))
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment