benchmarks/benchmark_latency.py · 973617ae02a4e8e6190674cf1cdb0c0803b65ae6 · OpenDAS / vllm_cscc · GitLab

Find file Blame History Permalink

[Speculative decoding][Re-take] Enable TP>1 speculative decoding (#4840) · 973617ae
Cody Yu authored May 16, 2024
```
Co-authored-by: Cade Daniel <edacih@gmail.com>
Co-authored-by: Cade Daniel <cade@anyscale.com>
```
973617ae

benchmark_latency.py 8.33 KB