Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
1325872e
Unverified
Commit
1325872e
authored
Oct 19, 2024
by
Nick Hill
Committed by
GitHub
Oct 18, 2024
Browse files
[Frontend] Avoid creating guided decoding LogitsProcessor unnecessarily (#9521)
parent
380e1863
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
2 deletions
+5
-2
vllm/sampling_params.py
vllm/sampling_params.py
+5
-2
No files found.
vllm/sampling_params.py
View file @
1325872e
...
...
@@ -49,14 +49,17 @@ class GuidedDecodingParams:
@
staticmethod
def
from_optional
(
json
:
Optional
[
Union
[
Dict
,
BaseModel
,
str
]],
json
:
Optional
[
Union
[
Dict
,
BaseModel
,
str
]]
=
None
,
regex
:
Optional
[
str
]
=
None
,
choice
:
Optional
[
List
[
str
]]
=
None
,
grammar
:
Optional
[
str
]
=
None
,
json_object
:
Optional
[
bool
]
=
None
,
backend
:
Optional
[
str
]
=
None
,
whitespace_pattern
:
Optional
[
str
]
=
None
,
)
->
"GuidedDecodingParams"
:
)
->
Optional
[
"GuidedDecodingParams"
]:
if
all
(
arg
is
None
for
arg
in
(
json
,
regex
,
choice
,
grammar
,
json_object
)):
return
None
# Extract json schemas from pydantic models
if
isinstance
(
json
,
(
BaseModel
,
type
(
BaseModel
))):
json
=
json
.
model_json_schema
()
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment