Skip to content

GitLab

Unverified Commit 3de3eadf authored by Wenhua Cheng's avatar Wenhua Cheng Committed by GitHub
Browse files

improve the robustness of parsing vlms config in AutoRound (#18894) 


Signed-off-by: default avatarwenhuach21 <wenhua.cheng@intel.com>
parent 3132290a
Show whitespace changes
Inline Side-by-side
Showing with 3 additions and 2 deletions
vllm/model_executor/layers/quantization/auto_round.py
View file @ 3de3eadf
...@@ -116,7 +116,8 @@ class AutoRoundConfig(QuantizationConfig): ...@@ -116,7 +116,8 @@ class AutoRoundConfig(QuantizationConfig):
quantized = True quantized = True
if self.block_name_to_quantize: if self.block_name_to_quantize:
quantized = any(name in layer_name quantized = any(
layer_name.startswith(name)
for name in self.block_name_to_quantize) for name in self.block_name_to_quantize)
elif isinstance(layer, ParallelLMHead): elif isinstance(layer, ParallelLMHead):
quantized = False quantized = False
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment