Skip to content

GitLab

Unverified Commit 39350031 authored by rollroll90's avatar rollroll90 Committed by GitHub
Browse files

Fix concatenate bug in benchmark serving script (#134)

parent 4970d798
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
benchmark/profile_serving.py
View file @ 39350031
...@@ -169,9 +169,9 @@ def main(tritonserver_addr: str, ...@@ -169,9 +169,9 @@ def main(tritonserver_addr: str,
session_id, _stats = res_que.get() session_id, _stats = res_que.get()
print(f'\n{"-" * 50}\n' print(f'\n{"-" * 50}\n'
f'session {session_id} stats: \n{_stats}\n{"-" * 50}\n') f'session {session_id} stats: \n{_stats}\n{"-" * 50}\n')
stats.append(_stats) stats.append(np.array(_stats))
stats = np.array(stats).reshape(-1, 3) stats = np.concatenate(stats).reshape(-1, 3)
first_token_latency_min = np.min(stats[:, 0], axis=0) first_token_latency_min = np.min(stats[:, 0], axis=0)
first_token_latency_max = np.max(stats[:, 0], axis=0) first_token_latency_max = np.max(stats[:, 0], axis=0)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment