Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
a7ea35aa
Unverified
Commit
a7ea35aa
authored
Mar 05, 2025
by
Tyler Michael Smith
Committed by
GitHub
Mar 05, 2025
Browse files
[Bugfix] Remove num_tokens_across_dp (#14302)
Signed-off-by:
Tyler Michael Smith
<
tyler@neuralmagic.com
>
parent
1e3e76b6
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
2 deletions
+1
-2
vllm/forward_context.py
vllm/forward_context.py
+1
-2
No files found.
vllm/forward_context.py
View file @
a7ea35aa
...
...
@@ -27,7 +27,6 @@ batchsize_forward_time: defaultdict = defaultdict(list)
@
dataclass
class
DPMetadata
:
num_tokens_across_dp
:
list
[
int
]
cu_tokens_across_dp_cpu
:
torch
.
Tensor
...
...
@@ -89,7 +88,7 @@ def set_forward_context(attn_metadata: Any,
from
vllm.distributed.parallel_state
import
get_dp_group
dist
.
all_reduce
(
num_tokens_tensor
,
group
=
get_dp_group
().
cpu_group
)
cu_tokens_across_dp_cpu
=
torch
.
cumsum
(
num_tokens_tensor
,
dim
=
0
)
dp_metadata
=
DPMetadata
(
num_tokens_across_dp
,
cu_tokens_across_dp_cpu
)
dp_metadata
=
DPMetadata
(
cu_tokens_across_dp_cpu
)
global
_forward_context
prev_context
=
_forward_context
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment