Skip to content

GitLab

Commit 56b9484f authored by jerrrrry's avatar jerrrrry
Browse files

Update benchmark_throughput.py

parent 1c5c33fa
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 3 deletions
2_env_check&model_download&llm_inference/scripts/benchmark_throughput.py
View file @ 56b9484f
......@@ -294,9 +294,9 @@ def run_vllm(
info["ttft_median"] = np.around(np.median(total_ttfts or 0),5)
info["ttft_p99"] = np.around(np.percentile(total_ttfts or 0, 99),5)
info["tpot_mean"] = np.around(np.mean(total_tpops),4)
info["tpot_median"] = np.around(np.median(total_tpops or 0),5)
info["tpot_p99"] = np.around(np.percentile(total_tpops or 0, 99),5)
info["tpop_mean"] = np.around(np.mean(total_tpops),4)
info["tpop_median"] = np.around(np.median(total_tpops or 0),5)
info["tpop_p99"] = np.around(np.percentile(total_tpops or 0, 99),5)
info["output_token_throughput_mean"] = np.around(np.mean(total_output_token_throughput),2)
info["output_token_throughput_median"] = np.around(np.median(total_output_token_throughput or 0),2)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment