Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
dea985ae
Unverified
Commit
dea985ae
authored
Mar 10, 2025
by
Roger Wang
Committed by
GitHub
Mar 10, 2025
Browse files
[V1][Bugfix] Fix handing of `second_per_grid_ts` for Qwen2-VL & Qwen2.5-VL (#14548)
Signed-off-by:
Roger Wang
<
ywang@roblox.com
>
parent
39be3035
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
10 additions
and
5 deletions
+10
-5
vllm/model_executor/layers/rotary_embedding.py
vllm/model_executor/layers/rotary_embedding.py
+10
-5
No files found.
vllm/model_executor/layers/rotary_embedding.py
View file @
dea985ae
...
@@ -923,14 +923,19 @@ class MRotaryEmbedding(RotaryEmbedding):
...
@@ -923,14 +923,19 @@ class MRotaryEmbedding(RotaryEmbedding):
def
get_input_positions
(
def
get_input_positions
(
input_tokens
:
List
[
int
],
input_tokens
:
List
[
int
],
hf_config
:
PretrainedConfig
,
hf_config
:
PretrainedConfig
,
image_grid_thw
:
Union
[
List
[
List
[
int
]],
torch
.
Tensor
],
image_grid_thw
:
Optional
[
Union
[
List
[
List
[
int
]],
torch
.
Tensor
]
]
,
video_grid_thw
:
Union
[
List
[
List
[
int
]],
torch
.
Tensor
],
video_grid_thw
:
Optional
[
Union
[
List
[
List
[
int
]],
torch
.
Tensor
]
]
,
second_per_grid_ts
:
Optional
[
List
[
float
]]
=
None
,
second_per_grid_ts
:
Optional
[
List
[
float
]],
context_len
:
int
=
0
,
context_len
:
int
=
0
,
seq_len
:
Optional
[
int
]
=
None
,
seq_len
:
Optional
[
int
]
=
None
,
)
->
Tuple
[
List
[
List
[
int
]],
int
]:
)
->
Tuple
[
List
[
List
[
int
]],
int
]:
"""Get mrope input positions and delta value."""
"""Get mrope input positions and delta value."""
image_grid_thw
=
[]
if
image_grid_thw
is
None
else
image_grid_thw
video_grid_thw
=
[]
if
video_grid_thw
is
None
else
video_grid_thw
second_per_grid_ts
=
[]
if
second_per_grid_ts
is
None
else
\
second_per_grid_ts
llm_positions
,
mrope_position_delta
=
\
llm_positions
,
mrope_position_delta
=
\
MRotaryEmbedding
.
get_input_positions_tensor
(
MRotaryEmbedding
.
get_input_positions_tensor
(
input_tokens
=
input_tokens
,
input_tokens
=
input_tokens
,
...
@@ -950,7 +955,7 @@ class MRotaryEmbedding(RotaryEmbedding):
...
@@ -950,7 +955,7 @@ class MRotaryEmbedding(RotaryEmbedding):
hf_config
:
PretrainedConfig
,
hf_config
:
PretrainedConfig
,
image_grid_thw
:
Union
[
List
[
List
[
int
]],
torch
.
Tensor
],
image_grid_thw
:
Union
[
List
[
List
[
int
]],
torch
.
Tensor
],
video_grid_thw
:
Union
[
List
[
List
[
int
]],
torch
.
Tensor
],
video_grid_thw
:
Union
[
List
[
List
[
int
]],
torch
.
Tensor
],
second_per_grid_ts
:
Optional
[
List
[
float
]
]
=
None
,
second_per_grid_ts
:
List
[
float
],
context_len
:
int
=
0
,
context_len
:
int
=
0
,
seq_len
:
Optional
[
int
]
=
None
,
seq_len
:
Optional
[
int
]
=
None
,
)
->
Tuple
[
torch
.
Tensor
,
int
]:
)
->
Tuple
[
torch
.
Tensor
,
int
]:
...
@@ -1006,7 +1011,7 @@ class MRotaryEmbedding(RotaryEmbedding):
...
@@ -1006,7 +1011,7 @@ class MRotaryEmbedding(RotaryEmbedding):
video_grid_thw
[
video_index
][
2
],
video_grid_thw
[
video_index
][
2
],
)
)
video_second_per_grid_t
=
1.0
video_second_per_grid_t
=
1.0
if
second_per_grid_ts
is
not
None
:
if
second_per_grid_ts
:
video_second_per_grid_t
=
second_per_grid_ts
[
video_index
]
video_second_per_grid_t
=
second_per_grid_ts
[
video_index
]
video_index
+=
1
video_index
+=
1
remain_videos
-=
1
remain_videos
-=
1
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment