Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
2755c34a
Unverified
Commit
2755c34a
authored
Apr 08, 2025
by
Russell Bryant
Committed by
GitHub
Apr 08, 2025
Browse files
[V1] Update structured output offline inference example (#15721)
Signed-off-by:
Russell Bryant
<
rbryant@redhat.com
>
parent
db104221
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
7 additions
and
12 deletions
+7
-12
examples/offline_inference/structured_outputs.py
examples/offline_inference/structured_outputs.py
+7
-12
No files found.
examples/offline_inference/structured_outputs.py
View file @
2755c34a
...
...
@@ -19,7 +19,7 @@ outputs = llm.generate(
print
(
outputs
[
0
].
outputs
[
0
].
text
)
# Guided decoding by Regex
guided_decoding_params
=
GuidedDecodingParams
(
regex
=
"\w+@\w+\.com
\n
"
)
guided_decoding_params
=
GuidedDecodingParams
(
regex
=
r
"\w+@\w+\.com\n"
)
sampling_params
=
SamplingParams
(
guided_decoding
=
guided_decoding_params
,
stop
=
[
"
\n
"
])
prompt
=
(
"Generate an email address for Alan Turing, who works in Enigma."
...
...
@@ -57,17 +57,12 @@ print(outputs[0].outputs[0].text)
# Guided decoding by Grammar
simplified_sql_grammar
=
"""
?start: select_statement
?select_statement: "SELECT " column_list " FROM " table_name
?column_list: column_name ("," column_name)*
?table_name: identifier
?column_name: identifier
?identifier: /[a-zA-Z_][a-zA-Z0-9_]*/
root ::= select_statement
select_statement ::= "SELECT " column " from " table " where " condition
column ::= "col_1 " | "col_2 "
table ::= "table_1 " | "table_2 "
condition ::= column "= " number
number ::= "1 " | "2 "
"""
guided_decoding_params
=
GuidedDecodingParams
(
grammar
=
simplified_sql_grammar
)
sampling_params
=
SamplingParams
(
guided_decoding
=
guided_decoding_params
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment