Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
1144a8ef
Unverified
Commit
1144a8ef
authored
May 01, 2025
by
NaLan ZeYu
Committed by
GitHub
Apr 30, 2025
Browse files
[Bugfix] Temporarily disable gptq_bitblas on ROCm (#17411)
Signed-off-by:
Yan Cangang
<
nalanzeyu@gmail.com
>
parent
08fb5587
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
6 additions
and
1 deletion
+6
-1
docs/source/features/quantization/supported_hardware.md
docs/source/features/quantization/supported_hardware.md
+1
-1
vllm/model_executor/layers/quantization/gptq_bitblas.py
vllm/model_executor/layers/quantization/gptq_bitblas.py
+5
-0
No files found.
docs/source/features/quantization/supported_hardware.md
View file @
1144a8ef
...
...
@@ -80,7 +80,7 @@ The table below shows the compatibility of various quantization implementations
*
✅︎
*
✅︎
*
✅︎
*
✅︎
*
❌
*
❌
*
❌
*
❌
...
...
vllm/model_executor/layers/quantization/gptq_bitblas.py
View file @
1144a8ef
...
...
@@ -25,6 +25,7 @@ from vllm.model_executor.parameter import (ChannelQuantScaleParameter,
PackedColumnParameter
,
PackedvLLMParameter
,
RowvLLMParameter
)
from
vllm.platforms
import
current_platform
from
vllm.scalar_type
import
scalar_types
logger
=
init_logger
(
__name__
)
...
...
@@ -191,6 +192,10 @@ class GPTQBitBLASConfig(QuantizationConfig):
sym
=
quant_config
.
get
(
"sym"
)
desc_act
=
quant_config
.
get
(
"desc_act"
)
# temporarily disable on ROCm platform
if
not
current_platform
.
is_cuda
():
return
False
# If we cannot find the info needed in the config, cannot convert.
if
(
num_bits
is
None
or
group_size
is
None
or
sym
is
None
or
desc_act
is
None
):
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment