Skip to content

GitLab

"Dockerfile" did not exist on "1cddb60bfae032b319a2ddcdf352c370ccd2aa3a"
Unverified Commit 470b4740 authored by Xu Song's avatar Xu Song Committed by GitHub
Browse files

Update bench_serving.py (#4454)

parent 26c372c1
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
python/sglang/bench_serving.py
View file @ 470b4740
...@@ -128,7 +128,7 @@ async def async_request_trt_llm( ...@@ -128,7 +128,7 @@ async def async_request_trt_llm(
timestamp = time.perf_counter() timestamp = time.perf_counter()
# First token # First token
if ttft == 0.0: if ttft == 0.0:
ttft = time.perf_counter() - st ttft = timestamp - st
output.ttft = ttft output.ttft = ttft
# Decoding phase # Decoding phase
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment