vllm/v1/worker/gpu_model_runner.py · b395b3b0a3166d17c75e74f4eaf0ff4b15f2554f · OpenDAS / vllm_cscc

[Disagg][Perf] Use CUDA event sync instead of blocking `tolist` to avoid... · b395b3b0

Zijing Liu authored Aug 25, 2025


[Disagg][Perf] Use CUDA event sync instead of blocking `tolist` to avoid unintentional copy ops blocking across different CUDA streams, improving disagg TTIT/TTFT (#22760)
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Zijing Liu <liuzijing2014@users.noreply.github.com>

b395b3b0

gpu_model_runner.py 148 KB

Replace gpu_model_runner.py