Commit cc946d6e authored by zhuwenwen's avatar zhuwenwen
Browse files

Merge branch 'v0.9.2-dev_tcopt' into 'v0.9.2-dev'

fix:更新 Marlin W16A16 MoE 中支持的断言消息

See merge request dcutoolkit/deeplearing/vllm!349
parents ae1b380d d77a3d5b
......@@ -240,7 +240,7 @@ def fused_experts_impl_w16a16_marlin(hidden_states: torch.Tensor,
assert hidden_states.dtype in [torch.bfloat16,torch.float16]
compute_type = hidden_states.dtype
assert use_lightop, (
"only BW and set LMSLIM_USE_LIGHTOP=1 support Marlin W16A16 MoE")
"only BW and set VLLM_USE_LIGHTOP=1 support Marlin W16A16 MoE")
num_tokens, K = hidden_states.shape
E, twoN, K_w1 = w1.shape
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment