Skip to content

GitLab

"vscode:/vscode.git/clone" did not exist on "b62439d71eaf154ea8dcd9d3f78b6e264e39e013"
Unverified Commit 05c50a82 authored by Roger Wang's avatar Roger Wang Committed by GitHub
Browse files

Minor bugfix on benchmark serving (#1005)

parent b568df5d
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 1 deletion
python/sglang/bench_serving.py
View file @ 05c50a82
...@@ -195,7 +195,8 @@ async def async_request_openai_completions( ...@@ -195,7 +195,8 @@ async def async_request_openai_completions(
output.ttft = ttft output.ttft = ttft
# Decoding phase # Decoding phase
output.itl.append(timestamp - most_recent_timestamp) else:
output.itl.append(timestamp - most_recent_timestamp)
most_recent_timestamp = timestamp most_recent_timestamp = timestamp
generated_text += data["choices"][0]["text"] generated_text += data["choices"][0]["text"]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment