fix bench serving bug (#4135)

3a391812 · Lzhang-hub · GitHub · 800bf018 · 3a391812
Unverified Commit 3a391812 authored Mar 06, 2025 by Lzhang-hub Committed by GitHub Mar 06, 2025
Hide whitespace changes
Inline Side-by-side

Showing with 1 addition and 1 deletion

python/sglang/bench_serving.py python/sglang/bench_serving.py +1 -1

No files found.
--- a/python/sglang/bench_serving.py
+++ b/python/sglang/bench_serving.py
@@ -220,7 +220,7 @@ async def async_request_openai_completions(

                                most_recent_timestamp = timestamp
                                generated_text += data["choices"][0]["text"]
-                                output_len = data.get("usage", {}).get(
+                                output_len = (data.get("usage") or {}).get(
                                    "completion_tokens", output_len
                                )