Unverified Commit a71e4765 authored by zifeitong's avatar zifeitong Committed by GitHub
Browse files

[Bugfix] Fix Qwen2.5-VL quantized model weights loading (#23512)


Signed-off-by: default avatarZifei Tong <zifeitong@gmail.com>
parent 39971db3
...@@ -135,7 +135,7 @@ class Qwen2_5_VLVideoPixelInputs(TypedDict): ...@@ -135,7 +135,7 @@ class Qwen2_5_VLVideoPixelInputs(TypedDict):
second_per_grid_ts: torch.Tensor second_per_grid_ts: torch.Tensor
""" """
The video time interval (in seconds) for each grid along the temporal The video time interval (in seconds) for each grid along the temporal
dimension in the 3D position IDs. Returned when `videos` is not `None`. dimension in the 3D position IDs. Returned when `videos` is not `None`.
""" """
...@@ -852,6 +852,10 @@ class Qwen2_5_VLForConditionalGeneration(nn.Module, SupportsMultiModal, ...@@ -852,6 +852,10 @@ class Qwen2_5_VLForConditionalGeneration(nn.Module, SupportsMultiModal,
SupportsLoRA, SupportsPP, SupportsLoRA, SupportsPP,
SupportsQuant): SupportsQuant):
packed_modules_mapping = {
"gate_up_proj": ["gate_proj", "up_proj"],
}
# To ensure correct weight loading and mapping. # To ensure correct weight loading and mapping.
hf_to_vllm_mapper = WeightsMapper( hf_to_vllm_mapper = WeightsMapper(
orig_to_new_prefix={ orig_to_new_prefix={
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment