vllm/spec_decode/batch_expansion.py · f256ebe4df6757d76f1f1642d7e110268a2f8190 · OpenDAS / vllm_cscc

[Feature] [Spec decode]: Enable MLPSpeculator/Medusa and `prompt_logprobs`... · 6116ca8c

Nicolò Lucchesi authored Jan 27, 2025


[Feature] [Spec decode]: Enable MLPSpeculator/Medusa and `prompt_logprobs` with ChunkedPrefill (#10132)
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: wallashss <wallashss@ibm.com>
Co-authored-by: wallashss <wallashss@ibm.com>

6116ca8c

batch_expansion.py 22.2 KB

Replace batch_expansion.py