vllm/spec_decode/spec_decode_worker.py · db66e018eaabcc5e5855e994b49931dbb4800ce1 · OpenDAS / vllm_cscc

[Bugfix] Fix for Spec model TP + Chunked Prefill (#10232) · db66e018

Murali Andoorveedu authored Nov 26, 2024


Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
Signed-off-by: Sourashis Roy <sroy@roblox.com>
Co-authored-by: Sourashis Roy <sroy@roblox.com>

db66e018

spec_decode_worker.py 53.3 KB

Replace spec_decode_worker.py