vllm/v1/worker/gpu_input_batch.py · f092153fbe349a9a1742940e3703bfcff6aa0a6d · OpenDAS / vllm_cscc · GitLab

Find file Blame History Permalink

[V1] Use more persistent buffers to optimize input preparation overheads (#11111) · f092153f
Woosuk Kwon authored Dec 11, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
f092153f

gpu_input_batch.py 10.5 KB