Skip to content

GitLab

Unverified Commit fa271613 authored by Yineng Zhang's avatar Yineng Zhang Committed by GitHub
Browse files

fix: use torch.sum for compatible (#2161)

parent 5652c565
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 2 deletions
python/sglang/srt/layers/triton_fused_moe/fused_moe.py
View file @ fa271613
...@@ -766,9 +766,10 @@ def fused_experts_impl( ...@@ -766,9 +766,10 @@ def fused_experts_impl(
use_int8_w8a16=use_int8_w8a16, use_int8_w8a16=use_int8_w8a16,
) )
ops.moe_sum( torch.sum(
intermediate_cache3.view(*intermediate_cache3.shape), intermediate_cache3.view(*intermediate_cache3.shape),
out_hidden_states[begin_chunk_idx:end_chunk_idx], dim=1,
out=out_hidden_states[begin_chunk_idx:end_chunk_idx],
) )
return out_hidden_states return out_hidden_states
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment