Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
a4398fbb
Unverified
Commit
a4398fbb
authored
Nov 03, 2025
by
Sophie du Couédic
Committed by
GitHub
Nov 03, 2025
Browse files
[Feature][Benchmarks] Support `inf` burstiness (#26941)
Signed-off-by:
Sophie du Couédic
<
sop@zurich.ibm.com
>
parent
2c19d967
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
7 additions
and
0 deletions
+7
-0
vllm/benchmarks/serve.py
vllm/benchmarks/serve.py
+7
-0
No files found.
vllm/benchmarks/serve.py
View file @
a4398fbb
...
...
@@ -189,9 +189,16 @@ async def get_request(
total_requests
,
request_rate
,
)
assert
current_request_rate
>
0.0
,
(
f
"Obtained non-positive request rate
{
current_request_rate
}
."
)
request_rates
.
append
(
current_request_rate
)
if
current_request_rate
==
float
(
"inf"
):
delay_ts
.
append
(
0
)
elif
burstiness
==
float
(
"inf"
):
# when burstiness tends to infinity, the delay time becomes constant
# and tends to the inverse of the request rate
delay_ts
.
append
(
1.0
/
current_request_rate
)
else
:
theta
=
1.0
/
(
current_request_rate
*
burstiness
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment