Skip to content

GitLab

Unverified Commit 70d268a3 authored by Roger Wang's avatar Roger Wang Committed by GitHub
Browse files

[Bugfix] Fix ITL recording in serving benchmark (#7372)

parent 249b8822
Show whitespace changes
Inline Side-by-side
Showing with 3 additions and 2 deletions
benchmarks/backend_request_func.py
View file @ 70d268a3
...@@ -276,6 +276,7 @@ async def async_request_openai_completions( ...@@ -276,6 +276,7 @@ async def async_request_openai_completions(
output.ttft = ttft output.ttft = ttft
# Decoding phase # Decoding phase
else:
output.itl.append(timestamp - output.itl.append(timestamp -
most_recent_timestamp) most_recent_timestamp)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment