Unverified Commit a71e4765 authored by zifeitong's avatar zifeitong Committed by GitHub
Browse files

[Bugfix] Fix Qwen2.5-VL quantized model weights loading (#23512)


Signed-off-by: default avatarZifei Tong <zifeitong@gmail.com>
parent 39971db3
......@@ -135,7 +135,7 @@ class Qwen2_5_VLVideoPixelInputs(TypedDict):
second_per_grid_ts: torch.Tensor
"""
The video time interval (in seconds) for each grid along the temporal
The video time interval (in seconds) for each grid along the temporal
dimension in the 3D position IDs. Returned when `videos` is not `None`.
"""
......@@ -852,6 +852,10 @@ class Qwen2_5_VLForConditionalGeneration(nn.Module, SupportsMultiModal,
SupportsLoRA, SupportsPP,
SupportsQuant):
packed_modules_mapping = {
"gate_up_proj": ["gate_proj", "up_proj"],
}
# To ensure correct weight loading and mapping.
hf_to_vllm_mapper = WeightsMapper(
orig_to_new_prefix={
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment