Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
1ece1ae8
Unverified
Commit
1ece1ae8
authored
Mar 08, 2024
by
TianYu GUO
Committed by
GitHub
Mar 07, 2024
Browse files
[Minor Fix] Fix comments in benchmark_serving (#3252)
parent
c59e120c
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
3 deletions
+3
-3
benchmarks/benchmark_serving.py
benchmarks/benchmark_serving.py
+3
-3
No files found.
benchmarks/benchmark_serving.py
View file @
1ece1ae8
...
@@ -12,7 +12,7 @@ On the server side, run one of the following commands:
...
@@ -12,7 +12,7 @@ On the server side, run one of the following commands:
On the client side, run:
On the client side, run:
python benchmarks/benchmark_serving.py
\
python benchmarks/benchmark_serving.py
\
--backend <backend>
\
--backend <backend>
\
--
tokenizer
<your_model> --dataset <target_dataset>
\
--
model
<your_model> --dataset <target_dataset>
\
--request-rate <request_rate>
--request-rate <request_rate>
"""
"""
import
argparse
import
argparse
...
@@ -171,10 +171,10 @@ async def benchmark(
...
@@ -171,10 +171,10 @@ async def benchmark(
else
:
else
:
raise
ValueError
(
f
"Unknown backend:
{
backend
}
"
)
raise
ValueError
(
f
"Unknown backend:
{
backend
}
"
)
pbar
=
None
if
disable_tqdm
else
tqdm
(
total
=
len
(
input_requests
))
print
(
f
"Traffic request rate:
{
request_rate
}
"
)
print
(
f
"Traffic request rate:
{
request_rate
}
"
)
pbar
=
None
if
disable_tqdm
else
tqdm
(
total
=
len
(
input_requests
))
benchmark_start_time
=
time
.
perf_counter
()
benchmark_start_time
=
time
.
perf_counter
()
tasks
=
[]
tasks
=
[]
async
for
request
in
get_request
(
input_requests
,
request_rate
):
async
for
request
in
get_request
(
input_requests
,
request_rate
):
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment