Minor bugfix on benchmark serving (#1005)

05c50a82 · Roger Wang · GitHub · b568df5d · 05c50a82
Unverified Commit 05c50a82 authored Aug 09, 2024 by Roger Wang Committed by GitHub Aug 10, 2024
Hide whitespace changes
Inline Side-by-side

Showing with 2 additions and 1 deletion

python/sglang/bench_serving.py python/sglang/bench_serving.py +2 -1

No files found.
--- a/python/sglang/bench_serving.py
+++ b/python/sglang/bench_serving.py
@@ -195,7 +195,8 @@ async def async_request_openai_completions(
                                    output.ttft = ttft

                                # Decoding phase
-                                output.itl.append(timestamp - most_recent_timestamp)
+                                else:
+                                    output.itl.append(timestamp - most_recent_timestamp)

                                most_recent_timestamp = timestamp
                                generated_text += data["choices"][0]["text"]