Skip to content

GitLab

"git@developer.sourcefind.cn:OpenDAS/ollama.git" did not exist on "8a8c7e7f8dcf4b71601e9912e6dbc12064d77a24"
Commit 56b9484f authored by jerrrrry's avatar jerrrrry
Browse files

Update benchmark_throughput.py

parent 1c5c33fa
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 3 deletions
2_env_check&model_download&llm_inference/scripts/benchmark_throughput.py
View file @ 56b9484f
...@@ -294,9 +294,9 @@ def run_vllm( ...@@ -294,9 +294,9 @@ def run_vllm(
info["ttft_median"] = np.around(np.median(total_ttfts or 0),5) info["ttft_median"] = np.around(np.median(total_ttfts or 0),5)
info["ttft_p99"] = np.around(np.percentile(total_ttfts or 0, 99),5) info["ttft_p99"] = np.around(np.percentile(total_ttfts or 0, 99),5)
info["tpot_mean"] = np.around(np.mean(total_tpops),4) info["tpop_mean"] = np.around(np.mean(total_tpops),4)
info["tpot_median"] = np.around(np.median(total_tpops or 0),5) info["tpop_median"] = np.around(np.median(total_tpops or 0),5)
info["tpot_p99"] = np.around(np.percentile(total_tpops or 0, 99),5) info["tpop_p99"] = np.around(np.percentile(total_tpops or 0, 99),5)
info["output_token_throughput_mean"] = np.around(np.mean(total_output_token_throughput),2) info["output_token_throughput_mean"] = np.around(np.mean(total_output_token_throughput),2)
info["output_token_throughput_median"] = np.around(np.median(total_output_token_throughput or 0),2) info["output_token_throughput_median"] = np.around(np.median(total_output_token_throughput or 0),2)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment