Skip to content

GitLab

Unverified Commit 61728884 authored by Mingyi's avatar Mingyi Committed by GitHub
Browse files

Fix benchmark latency (#1007)

parent 9cf0a5ba
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
python/sglang/bench_latency.py
View file @ 61728884
......@@ -332,6 +332,7 @@ def latency_test(
)
# Warm up
rank_print("Warmup ...")
latency_test_run_once(
bench_args.run_name,
model_runner,
......@@ -341,6 +342,7 @@ def latency_test(
bench_args.input_len[0],
4, # shorter decoding to speed up the warmup
)
rank_print("Benchmark ...")
# Run the sweep
result_list = []
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment