vllm/engine/llm_engine.py · 6116ca8cd79b642c64f4ae6f050a6bc12b96d037 · OpenDAS / vllm_cscc

[Feature] [Spec decode]: Enable MLPSpeculator/Medusa and `prompt_logprobs`... · 6116ca8c

Nicolò Lucchesi authored Jan 27, 2025


[Feature] [Spec decode]: Enable MLPSpeculator/Medusa and `prompt_logprobs` with ChunkedPrefill (#10132)
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: wallashss <wallashss@ibm.com>
Co-authored-by: wallashss <wallashss@ibm.com>

6116ca8c

llm_engine.py 86.5 KB

Replace llm_engine.py