Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
d544d141
Unverified
Commit
d544d141
authored
Apr 10, 2025
by
Chenyaaang
Committed by
GitHub
Apr 11, 2025
Browse files
update benchmark_serving_structured_output to include auto backend (#16438)
Signed-off-by:
Chenyaaang
<
chenyangli@google.com
>
parent
3e397a94
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
9 additions
and
7 deletions
+9
-7
benchmarks/benchmark_serving_structured_output.py
benchmarks/benchmark_serving_structured_output.py
+9
-7
No files found.
benchmarks/benchmark_serving_structured_output.py
View file @
d544d141
...
...
@@ -11,7 +11,7 @@ On the client side, run:
--model <your_model> \
--dataset json \
--structured-output-ratio 1.0 \
--structured-output-backend
xgrammar
\
--structured-output-backend
auto
\
--request-rate 10 \
--num-prompts 1000
...
...
@@ -997,12 +997,14 @@ if __name__ == "__main__":
type
=
float
,
default
=
1.0
,
help
=
"Ratio of Structured Outputs requests"
)
parser
.
add_argument
(
"--structured-output-backend"
,
type
=
str
,
choices
=
[
"outlines"
,
"lm-format-enforcer"
,
"xgrammar"
,
"guidance"
],
default
=
"xgrammar"
,
help
=
"Backend to use for structured outputs"
)
parser
.
add_argument
(
"--structured-output-backend"
,
type
=
str
,
choices
=
[
"outlines"
,
"lm-format-enforcer"
,
"xgrammar"
,
"guidance"
,
"auto"
],
default
=
"auto"
,
help
=
"Backend to use for structured outputs"
)
args
=
parser
.
parse_args
()
main
(
args
)
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment