Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
69ecaa7c
Unverified
Commit
69ecaa7c
authored
Apr 08, 2025
by
Alex Brooks
Committed by
GitHub
Apr 08, 2025
Browse files
[Misc] Add warning for multimodal data in LLM.beam_search (#16241)
Signed-off-by:
Alex-Brooks
<
Alex.Brooks@ibm.com
>
parent
7f00899f
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
10 additions
and
0 deletions
+10
-0
vllm/entrypoints/llm.py
vllm/entrypoints/llm.py
+10
-0
No files found.
vllm/entrypoints/llm.py
View file @
69ecaa7c
...
...
@@ -536,6 +536,16 @@ class LLM:
tokenizer
.
eos_token_id
,
length_penalty
)
# TODO - fix handling of multimodal data for beam search; we pass it
# through in the async version on the abstract EngineClient, but not
# here.
if
any
(
"multi_modal_data"
in
prompt
and
prompt
[
"multi_modal_data"
]
is
not
None
for
prompt
in
prompts
):
logger
.
warning
(
"Multimodal data appears to have been provided, but is not"
" currently being passed through in LLM.beam_search()!"
)
tokenizer
=
self
.
get_tokenizer
()
# generate 2 * beam_width candidates at each step
# following the huggingface transformers implementation
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment