Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
change
sglang
Commits
0242bb9c
"docs/source/en/installation.md" did not exist on "12fd0736dcc51f77c52130ab10177d0c1d5a29d9"
Unverified
Commit
0242bb9c
authored
Aug 04, 2025
by
Ke Bao
Committed by
GitHub
Aug 03, 2025
Browse files
Fix triton kernels topk with keyword arguments (#8732)
parent
760286e3
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
22 additions
and
3 deletions
+22
-3
python/sglang/srt/layers/moe/topk.py
python/sglang/srt/layers/moe/topk.py
+22
-3
No files found.
python/sglang/srt/layers/moe/topk.py
View file @
0242bb9c
...
...
@@ -183,12 +183,15 @@ class TopK(CustomOp):
*
,
num_token_non_padded
:
Optional
[
torch
.
Tensor
]
=
None
,
expert_location_dispatch_info
:
Optional
[
ExpertLocationDispatchInfo
]
=
None
,
sm_first
:
bool
=
False
,
# only used for triton kernels topk
)
->
TopKOutput
:
if
self
.
use_triton_kernels
:
routing_data
,
gather_idx
,
scatter_idx
=
routing
(
router_logits
,
self
.
top_k
,
self
.
renormalize
return
triton_kernels_topk
(
router_logits
=
router_logits
,
topk
=
self
.
top_k
,
renormalize
=
self
.
renormalize
,
sm_first
=
sm_first
,
)
return
TritonKernelTopKOutput
(
routing_data
,
gather_idx
,
scatter_idx
)
else
:
torch_native
=
False
return
select_experts
(
...
...
@@ -644,6 +647,22 @@ def biased_grouped_topk_cpu(
)
def
triton_kernels_topk
(
router_logits
:
torch
.
Tensor
,
topk
:
int
,
renormalize
:
bool
=
False
,
sm_first
:
bool
=
False
,
)
->
TritonKernelTopKOutput
:
"""Top-K routing for Triton kernels MoE."""
assert
not
renormalize
,
"Triton kernels topk doesn't support renormalize"
routing_data
,
gather_idx
,
scatter_idx
=
routing
(
logits
=
router_logits
,
n_expts_act
=
topk
,
sm_first
=
sm_first
,
)
return
TritonKernelTopKOutput
(
routing_data
,
gather_idx
,
scatter_idx
)
if
_is_cpu
and
_is_cpu_amx_available
:
biased_grouped_topk
=
biased_grouped_topk_cpu
grouped_topk
=
grouped_topk_cpu
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment