Skip to content

GitLab

"...text-generation-inference.git" did not exist on "d71237fc8b6fd82407b0e27b58cd32c61f2ab6cb"
Unverified Commit 5dd80d37 authored by Woosuk Kwon's avatar Woosuk Kwon Committed by GitHub
Browse files

Fix latency benchmark script (#2035)

parent beeee69b
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
benchmarks/benchmark_latency.py
View file @ 5dd80d37
...@@ -72,7 +72,7 @@ def main(args: argparse.Namespace): ...@@ -72,7 +72,7 @@ def main(args: argparse.Namespace):
# Benchmark. # Benchmark.
latencies = [] latencies = []
for _ in tqdm(range(args.num_iters), desc="Profiling iterations"): for _ in tqdm(range(args.num_iters), desc="Profiling iterations"):
latencies.append(run_to_completion(profile=False)) latencies.append(run_to_completion(profile_dir=None))
print(f'Avg latency: {np.mean(latencies)} seconds') print(f'Avg latency: {np.mean(latencies)} seconds')
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment