vllm/spec_decode/spec_decode_worker.py · b12518d3cf4326dfcd10a09780913b86c19fcf1a · OpenDAS / vllm_cscc

[Model] MLPSpeculator speculative decoding support (#4947) · b12518d3

Joshua Rosenkranz authored Jun 20, 2024


Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Davis Wertheimer <Davis.Wertheimer@ibm.com>

b12518d3

spec_decode_worker.py 28.1 KB

Replace spec_decode_worker.py