Skip to content

GitLab

Unverified Commit 4611af16 authored by Alex Brooks's avatar Alex Brooks Committed by GitHub
Browse files

[Bugfix] Add Quant Config to Llava Next Projector (#34847) 


Signed-off-by: default avatarAlex Brooks <albrooks@redhat.com>
parent ad5aa6bd
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
vllm/model_executor/models/llava_next.py
View file @ 4611af16
...@@ -285,6 +285,8 @@ class LlavaNextForConditionalGeneration(nn.Module, SupportsMultiModal, SupportsP ...@@ -285,6 +285,8 @@ class LlavaNextForConditionalGeneration(nn.Module, SupportsMultiModal, SupportsP
text_hidden_size=config.text_config.hidden_size, text_hidden_size=config.text_config.hidden_size,
projector_hidden_act=config.projector_hidden_act, projector_hidden_act=config.projector_hidden_act,
multimodal_projector_bias=config.multimodal_projector_bias, multimodal_projector_bias=config.multimodal_projector_bias,
quant_config=quant_config,
prefix=maybe_prefix(prefix, "multi_modal_projector"),
) )
with self._mark_language_model(vllm_config): with self._mark_language_model(vllm_config):
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment