Commit a3488ab0 authored by zhuwenwen's avatar zhuwenwen
Browse files

Merge branch 'v0.11.0-dev-wm-0122' into 'v0.11.0-dev'

[fix]解决dp+graph+naive all2all dispatch时的维度不匹配问题

See merge request dcutoolkit/deeplearing/vllm!384
parents 3cd3d1e6 d2c20c55
......@@ -3026,6 +3026,8 @@ class GPUModelRunner(LoRAModelRunnerMixin, KVConnectorModelRunnerMixin):
num_tokens_across_dp = num_tokens_after_padding
num_tokens_after_padding = int(num_tokens_after_padding[0].item())
num_tokens = num_tokens_after_padding
attn_metadata: Optional[PerLayerAttnMetadata] = None
# If force_attention is True, we always capture attention. Otherwise,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment