vllm/spec_decode/top1_proposer.py · 44cc76610d0b23ce5d609867f6dae7e033dee818 · OpenDAS / vllm_cscc

[Model] MLPSpeculator speculative decoding support (#4947) · b12518d3

Joshua Rosenkranz authored Jun 20, 2024


Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Davis Wertheimer <Davis.Wertheimer@ibm.com>

b12518d3

top1_proposer.py 11.6 KB

Replace top1_proposer.py