vllm/worker/model_runner.py · 0590ec3fd9857063c43c80df281e24c16c51b2ec · OpenDAS / vllm_cscc

"vllm/vscode:/vscode.git/clone" did not exist on "9855b99502c7537db5ef018129e603650800ac46"

[Core] Implement disagg prefill by StatelessProcessGroup (#10502) · 0590ec3f

Kuntai Du authored Dec 01, 2024



This PR provides initial support for single-node disaggregated prefill in 1P1D scenario.
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: YaoJiayi <120040070@link.cuhk.edu.cn>

0590ec3f

model_runner.py 88.5 KB

Replace model_runner.py