Unverified Commit 315b860a authored by Lucia Fang's avatar Lucia Fang Committed by GitHub
Browse files

[bugfix]fix empty prompts for async-engine mode in benchmark throughput (#27494)


Signed-off-by: default avatarLucia Fang <fanglu@fb.com>
parent 87c41c26
...@@ -221,6 +221,7 @@ async def run_vllm_async( ...@@ -221,6 +221,7 @@ async def run_vllm_async(
detokenize=not disable_detokenize, detokenize=not disable_detokenize,
) )
) )
prompts.append(prompt)
lora_requests.append(request.lora_request) lora_requests.append(request.lora_request)
generators = [] generators = []
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment