Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
4d022cbc
Unverified
Commit
4d022cbc
authored
Apr 11, 2025
by
Nicolò Lucchesi
Committed by
GitHub
Apr 11, 2025
Browse files
[TPU][V1] Make `--disable_chunked_mm_input` mandatory for serving MM models (#16483)
Signed-off-by:
NickLucche
<
nlucches@redhat.com
>
parent
70de35a8
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
7 additions
and
0 deletions
+7
-0
vllm/platforms/tpu.py
vllm/platforms/tpu.py
+7
-0
No files found.
vllm/platforms/tpu.py
View file @
4d022cbc
...
@@ -120,6 +120,13 @@ class TpuPlatform(Platform):
...
@@ -120,6 +120,13 @@ class TpuPlatform(Platform):
assert
not
vllm_config
.
speculative_config
,
(
assert
not
vllm_config
.
speculative_config
,
(
"Speculative decoding is not yet supported for TPU backend"
)
"Speculative decoding is not yet supported for TPU backend"
)
if
scheduler_config
.
is_multimodal_model
and
not
\
scheduler_config
.
disable_chunked_mm_input
:
logger
.
warning
(
"TPU does not support running Multimodal models"
\
" without setting `--disable_chunked_mm_input`. "
\
"Forcing --disable_chunked_mm_input."
)
scheduler_config
.
disable_chunked_mm_input
=
True
@
classmethod
@
classmethod
def
is_pin_memory_available
(
cls
):
def
is_pin_memory_available
(
cls
):
logger
.
warning
(
"Pin memory is not supported on TPU."
)
logger
.
warning
(
"Pin memory is not supported on TPU."
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment