Skip to content

GitLab

Commit 3d4721f2 authored by yangql1's avatar yangql1
Browse files

修复CompressedTensors的w8a16的支持

parent 3249f6cc
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
vllm/model_executor/layers/quantization/compressed_tensors/compressed_tensors.py
View file @ 3d4721f2
...@@ -730,7 +730,7 @@ class CompressedTensorsLinearMethod(LinearMethodBase): ...@@ -730,7 +730,7 @@ class CompressedTensorsLinearMethod(LinearMethodBase):
) )
if weights_scheme is not None: if weights_scheme is not None:
num_bits = weights_scheme.num_bits num_bits = weights_scheme.num_bits
if num_bits == 4: if isinstance(layer.scheme, CompressedTensorsWNA16):
return layer.scheme.process_weights_after_loading(layer) return layer.scheme.process_weights_after_loading(layer)
n=layer.weight.shape[0] n=layer.weight.shape[0]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment