Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
change
sglang
Commits
89caf7a3
Unverified
Commit
89caf7a3
authored
Aug 01, 2025
by
Trevor Morris
Committed by
GitHub
Aug 01, 2025
Browse files
[bugfix] Apply routed scaling factor to cutlass_fused_experts_fp8 (#8688)
parent
b27b1191
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
1 deletion
+5
-1
python/sglang/srt/layers/quantization/fp8.py
python/sglang/srt/layers/quantization/fp8.py
+5
-1
No files found.
python/sglang/srt/layers/quantization/fp8.py
View file @
89caf7a3
...
@@ -1039,7 +1039,7 @@ class Fp8MoEMethod(FusedMoEMethodBase):
...
@@ -1039,7 +1039,7 @@ class Fp8MoEMethod(FusedMoEMethodBase):
from
sglang.srt.layers.moe.cutlass_moe
import
cutlass_fused_experts_fp8
from
sglang.srt.layers.moe.cutlass_moe
import
cutlass_fused_experts_fp8
topk_weights
,
topk_ids
,
_
=
topk_output
topk_weights
,
topk_ids
,
_
=
topk_output
return
cutlass_fused_experts_fp8
(
output
=
cutlass_fused_experts_fp8
(
x
,
x
,
layer
.
w13_weight
.
transpose
(
1
,
2
),
layer
.
w13_weight
.
transpose
(
1
,
2
),
layer
.
w2_weight
.
transpose
(
1
,
2
),
layer
.
w2_weight
.
transpose
(
1
,
2
),
...
@@ -1062,6 +1062,10 @@ class Fp8MoEMethod(FusedMoEMethodBase):
...
@@ -1062,6 +1062,10 @@ class Fp8MoEMethod(FusedMoEMethodBase):
self
.
problem_sizes2
,
self
.
problem_sizes2
,
use_fp8_blockscale
=
True
,
use_fp8_blockscale
=
True
,
)
)
# TODO: Fuse into select_experts
if
routed_scaling_factor
is
not
None
:
output
*=
routed_scaling_factor
return
output
# Expert fusion with FP8 quantization
# Expert fusion with FP8 quantization
return
fused_experts
(
return
fused_experts
(
x
,
x
,
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment