Unverified Commit e6a226ef authored by Wentao Ye's avatar Wentao Ye Committed by GitHub
Browse files

[Bug] Fix AttributeError: 'QKVParallelLinear' object has no attribute 'orig_dtype' (#25958)


Signed-off-by: default avataryewentao256 <zhyanwentao@126.com>
parent a2e6fa7e
......@@ -66,6 +66,7 @@ class CompressedTensorsW8A8Fp8(CompressedTensorsScheme):
output_size_per_partition = sum(output_partition_sizes)
layer.logical_widths = output_partition_sizes
layer.weight_block_size = None
layer.orig_dtype = params_dtype
if self.strategy == QuantizationStrategy.BLOCK:
assert self.weight_block_size is not None
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment