Skip to content

GitLab

Unverified Commit 84bee4bd authored by Jee Jee Li's avatar Jee Jee Li Committed by GitHub
Browse files

[Misc] Improve the readability of BNB error messages (#12320) 


Signed-off-by: default avatarJee Jee Li <pandaleefree@gmail.com>
parent fc66dee7
Show whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
vllm/model_executor/model_loader/loader.py
View file @ 84bee4bd
...@@ -1076,8 +1076,8 @@ class BitsAndBytesModelLoader(BaseModelLoader): ...@@ -1076,8 +1076,8 @@ class BitsAndBytesModelLoader(BaseModelLoader):
# weight tensor. So TP does not work with pre_quantized bnb models. # weight tensor. So TP does not work with pre_quantized bnb models.
if pre_quant and get_tensor_model_parallel_world_size() > 1: if pre_quant and get_tensor_model_parallel_world_size() > 1:
raise ValueError( raise ValueError(
"Prequant BitsAndBytes models with TP is not supported." "Prequant BitsAndBytes models with tensor parallelism is not "
"Please try with PP.") "supported. Please try with pipeline parallelism.")
load_8bit = False load_8bit = False
if pre_quant: if pre_quant:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment