Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
change
sglang
Commits
94d26d85
Unverified
Commit
94d26d85
authored
Oct 15, 2025
by
strgrb
Committed by
GitHub
Oct 14, 2025
Browse files
use non_blocking h2d in ForwardBatch.prepare_mlp_sync_batch. (#11605)
parent
9e8a15a7
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
3 deletions
+2
-3
python/sglang/srt/model_executor/forward_batch_info.py
python/sglang/srt/model_executor/forward_batch_info.py
+2
-3
No files found.
python/sglang/srt/model_executor/forward_batch_info.py
View file @
94d26d85
...
...
@@ -734,9 +734,8 @@ class ForwardBatch:
self
.
encoder_lens
=
self
.
_pad_tensor_to_size
(
self
.
encoder_lens
,
bs
)
self
.
positions
=
self
.
_pad_tensor_to_size
(
self
.
positions
,
num_tokens
)
self
.
global_num_tokens_cpu
=
global_num_tokens
self
.
global_num_tokens_gpu
=
self
.
global_num_tokens_gpu
.
new_tensor
(
global_num_tokens
)
global_num_tokens_pinned
=
torch
.
tensor
(
global_num_tokens
,
pin_memory
=
True
)
self
.
global_num_tokens_gpu
.
copy_
(
global_num_tokens_pinned
,
non_blocking
=
True
)
if
self
.
mrope_positions
is
not
None
:
self
.
mrope_positions
=
self
.
_pad_tensor_to_size
(
self
.
mrope_positions
,
bs
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment