Skip to content

GitLab

Unverified Commit 70d268a3 authored by Roger Wang's avatar Roger Wang Committed by GitHub
Browse files

[Bugfix] Fix ITL recording in serving benchmark (#7372)

parent 249b8822
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 2 deletions
benchmarks/backend_request_func.py
View file @ 70d268a3
...@@ -276,8 +276,9 @@ async def async_request_openai_completions( ...@@ -276,8 +276,9 @@ async def async_request_openai_completions(
output.ttft = ttft output.ttft = ttft
# Decoding phase # Decoding phase
output.itl.append(timestamp - else:
most_recent_timestamp) output.itl.append(timestamp -
most_recent_timestamp)
most_recent_timestamp = timestamp most_recent_timestamp = timestamp
generated_text += data["choices"][0]["text"] generated_text += data["choices"][0]["text"]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment