Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
eb1051fb
Unverified
Commit
eb1051fb
authored
Dec 08, 2025
by
Ye (Charlotte) Qi
Committed by
GitHub
Dec 08, 2025
Browse files
[ROCm] Guard group quant RMS norm fusion patterns (#30239)
parent
80433e22
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
19 additions
and
17 deletions
+19
-17
vllm/compilation/fusion.py
vllm/compilation/fusion.py
+19
-17
No files found.
vllm/compilation/fusion.py
View file @
eb1051fb
...
@@ -490,6 +490,8 @@ class RMSNormQuantFusionPass(VllmPatternMatcherPass):
...
@@ -490,6 +490,8 @@ class RMSNormQuantFusionPass(VllmPatternMatcherPass):
# as the latter is a subset of the former in torch ops
# as the latter is a subset of the former in torch ops
for
epsilon
in
[
1e-5
,
1e-6
]:
for
epsilon
in
[
1e-5
,
1e-6
]:
# Fuse fused_add_rms_norm + fp8 group quant
# Fuse fused_add_rms_norm + fp8 group quant
# Only register group quant patterns on CUDA where the C++ op exists
if
current_platform
.
is_cuda
():
FusedAddRMSNormGroupQuantPattern
(
FusedAddRMSNormGroupQuantPattern
(
epsilon
,
FP8_DTYPE
,
group_shape
=
GroupShape
(
1
,
128
)
epsilon
,
FP8_DTYPE
,
group_shape
=
GroupShape
(
1
,
128
)
).
register
(
self
.
patterns
)
).
register
(
self
.
patterns
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment