Skip to content

GitLab

Unverified Commit c1b57855 authored by Nicolò Lucchesi's avatar Nicolò Lucchesi Committed by GitHub
Browse files

[TPU][V1] Use `language_model` interface for getting text backbone in MM (#16410) 


Signed-off-by: default avatarNickLucche <nlucches@redhat.com>
parent 83b824c8
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 2 deletions
vllm/v1/worker/tpu_model_runner.py
View file @ c1b57855
...@@ -972,8 +972,7 @@ class TPUModelRunner: ...@@ -972,8 +972,7 @@ class TPUModelRunner:
def reset_dynamo_cache(self): def reset_dynamo_cache(self):
if self.is_multimodal_model: if self.is_multimodal_model:
assert hasattr(self.model, "language_model") compiled_model = self.model.get_language_model().model
compiled_model = self.model.language_model.model
else: else:
compiled_model = self.model.model compiled_model = self.model.model
if isinstance(compiled_model, TorchCompileWrapperWithCustomDispatcher): if isinstance(compiled_model, TorchCompileWrapperWithCustomDispatcher):
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment