benchmarks/benchmark_serving.py · 0640f227c9a5fb1e7193792eb36ff48d152af099 · OpenDAS / vllm_cscc · GitLab

Find file Blame History Permalink

[MISC] Replace input token throughput with total token throughput (#8164) · 77d9e514
Cody Yu authored Sep 04, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
77d9e514

benchmark_serving.py 28.8 KB