vllm/forward_context.py · d6fd3a33b863003929fc3eef5dd9828219e04ab0 · OpenDAS / vllm_cscc

[Misc] reuse num_tokens_across_dp of get_dp_padding to avoid unnecessary dp... · d6fd3a33

zhrrr authored Jun 02, 2025


[Misc] reuse num_tokens_across_dp of get_dp_padding to avoid unnecessary dp all reduce in set_forward_context (#18935)
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>

d6fd3a33

forward_context.py 7 KB

Replace forward_context.py