Skip to content

GitLab

Unverified Commit dcf2a590 authored by Jerry Zhang's avatar Jerry Zhang Committed by GitHub
Browse files

Allow torchao quantization in SiglipMLP (#15575)

parent 54aa6194
Hide whitespace changes
Inline Side-by-side
Showing with 4 additions and 2 deletions
vllm/model_executor/models/siglip.py
View file @ dcf2a590
...@@ -208,8 +208,10 @@ class SiglipMLP(nn.Module): ...@@ -208,8 +208,10 @@ class SiglipMLP(nn.Module):
self.config = config self.config = config
self.activation_fn = get_act_fn(config.hidden_act) self.activation_fn = get_act_fn(config.hidden_act)
# Special handling for BNB quantization # Special handling for BNB and torchao quantization
if quant_config and quant_config.get_name() == "bitsandbytes": if quant_config and quant_config.get_name() in [
"bitsandbytes", "torchao"
]:
quantizable = True quantizable = True
else: else:
# For other quantization, we require the hidden size to be a # For other quantization, we require the hidden size to be a
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment