vllm/engine/async_llm_engine.py · 8a1415cf776b2b902f6429ecfc325877b57cbefe · OpenDAS / vllm_cscc

[BugFix]: fix engine timeout due to request abort (#6255) · 546b101f

pushan authored Jul 11, 2024


Signed-off-by: yatta zhang <ytzhang01@foxmail.com>
Signed-off-by: zhangyuntao.dev <zhangyuntao.dev@bytedance.com>
Co-authored-by: zhangyuntao.dev <zhangyuntao.dev@bytedance.com>

546b101f

async_llm_engine.py 38.3 KB

Replace async_llm_engine.py