Unverified Commit a71e4765 authored by zifeitong's avatar zifeitong Committed by GitHub
Browse files

[Bugfix] Fix Qwen2.5-VL quantized model weights loading (#23512)


Signed-off-by: default avatarZifei Tong <zifeitong@gmail.com>
parent 39971db3
...@@ -852,6 +852,10 @@ class Qwen2_5_VLForConditionalGeneration(nn.Module, SupportsMultiModal, ...@@ -852,6 +852,10 @@ class Qwen2_5_VLForConditionalGeneration(nn.Module, SupportsMultiModal,
SupportsLoRA, SupportsPP, SupportsLoRA, SupportsPP,
SupportsQuant): SupportsQuant):
packed_modules_mapping = {
"gate_up_proj": ["gate_proj", "up_proj"],
}
# To ensure correct weight loading and mapping. # To ensure correct weight loading and mapping.
hf_to_vllm_mapper = WeightsMapper( hf_to_vllm_mapper = WeightsMapper(
orig_to_new_prefix={ orig_to_new_prefix={
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment