vllm/engine/arg_utils.py · 3bcb8c75da56e39cfd6e440402cf043a3d470b33 · OpenDAS / vllm_cscc

[Core] Reduce TTFT with concurrent partial prefills (#10235) · 3bcb8c75

Joe Runde authored Feb 14, 2025


Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>

3bcb8c75

arg_utils.py 65.2 KB

Replace arg_utils.py