Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
change
sglang
Commits
f4674df6
Unverified
Commit
f4674df6
authored
Jul 25, 2025
by
ZhichenJiang
Committed by
GitHub
Jul 24, 2025
Browse files
support idle batch for TBO (#8233)
parent
d40846d4
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
8 additions
and
5 deletions
+8
-5
python/sglang/srt/two_batch_overlap.py
python/sglang/srt/two_batch_overlap.py
+8
-5
No files found.
python/sglang/srt/two_batch_overlap.py
View file @
f4674df6
...
...
@@ -341,15 +341,18 @@ class TboDPAttentionPreparer:
@
staticmethod
def
_compute_global_forward_mode
(
forward_modes
):
converted_forward_modes
=
[
ForwardMode
.
DECODE
.
value
if
x
==
ForwardMode
.
IDLE
.
value
else
x
for
x
in
forward_modes
forward_modes_excluding_idle
=
[
x
for
x
in
forward_modes
if
x
!=
ForwardMode
.
IDLE
.
value
]
if
not
forward_modes_excluding_idle
:
return
ForwardMode
.
IDLE
,
False
forward_mode_agree
=
TboDPAttentionPreparer
.
_is_all_same
(
converted_
forward_modes
forward_modes
_excluding_idle
)
global_forward_mode
=
(
ForwardMode
(
converted_
forward_modes
[
0
])
if
forward_mode_agree
else
None
ForwardMode
(
forward_modes
_excluding_idle
[
0
])
if
forward_mode_agree
else
None
)
return
global_forward_mode
,
forward_mode_agree
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment