Unverified Commit 0b225fb7 authored by Komal Kumar Teru's avatar Komal Kumar Teru Committed by GitHub
Browse files

[Misc] skip target model mm emb in draft proposal step when draft is text-only (#33437)


Signed-off-by: default avatarkkt-cohere <komal@cohere.com>
parent 46b4a027
......@@ -4070,7 +4070,7 @@ class GPUModelRunner(
else:
target_hidden_states = hidden_states[:total_num_tokens]
if self.supports_mm_inputs:
if self.supports_mm_inputs and self.drafter.supports_mm_inputs:
mm_embed_inputs = self._gather_mm_embeddings(
scheduler_output,
shift_computed_tokens=1,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment