Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
77d24c4b
Unverified
Commit
77d24c4b
authored
Mar 22, 2026
by
Wentao Ye
Committed by
GitHub
Mar 22, 2026
Browse files
[Bug] Fix fp8 deepgemm batch invariant (#37718)
Signed-off-by:
yewentao256
<
zhyanwentao@126.com
>
parent
b3e84601
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
0 deletions
+5
-0
vllm/model_executor/layers/quantization/utils/fp8_utils.py
vllm/model_executor/layers/quantization/utils/fp8_utils.py
+5
-0
No files found.
vllm/model_executor/layers/quantization/utils/fp8_utils.py
View file @
77d24c4b
...
...
@@ -305,6 +305,11 @@ def _flashinfer_fp8_blockscale_gemm_impl(
)
return
output
from
vllm.model_executor.layers.batch_invariant
import
vllm_is_batch_invariant
if
vllm_is_batch_invariant
():
return
run_deepgemm
(
input
,
weight
,
weight_scale
)
condition
=
input
.
shape
[
0
]
<
32
# PyTorch's torch.compile cannot handle input-dependent control flow in standard
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment