fix: resolve the fp8 bug introduced by vLLM 0.5.5 (#1276)

52cefdbf · Yineng Zhang · GitHub · 51c554d8 · 52cefdbf
Unverified Commit 52cefdbf authored Sep 01, 2024 by Yineng Zhang Committed by GitHub Sep 01, 2024
Hide whitespace changes
Inline Side-by-side

Showing with 1 addition and 1 deletion

python/sglang/srt/models/deepseek_v2.py python/sglang/srt/models/deepseek_v2.py +1 -1

No files found.
--- a/python/sglang/srt/models/deepseek_v2.py
+++ b/python/sglang/srt/models/deepseek_v2.py
@@ -695,7 +695,7 @@ class DeepseekV2ForCausalLM(nn.Module):
                    weight_loader(
                        param,
                        loaded_weight,
-                        weight_name,
+                        name,
                        shard_id=shard_id,
                        expert_id=expert_id,
                    )