Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
47f670b0
Unverified
Commit
47f670b0
authored
Sep 17, 2025
by
samzong
Committed by
GitHub
Sep 17, 2025
Browse files
[Docs] improve code formatting and comments for eliminate griffe build warning. (#25010)
Signed-off-by:
samzong
<
samzong.lu@gmail.com
>
parent
dd6a910a
Changes
3
Show whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
20 additions
and
14 deletions
+20
-14
vllm/benchmarks/serve.py
vllm/benchmarks/serve.py
+1
-1
vllm/distributed/eplb/eplb_state.py
vllm/distributed/eplb/eplb_state.py
+5
-4
vllm/distributed/eplb/rebalance_algo.py
vllm/distributed/eplb/rebalance_algo.py
+14
-9
No files found.
vllm/benchmarks/serve.py
View file @
47f670b0
vllm/distributed/eplb/eplb_state.py
View file @
47f670b0
...
@@ -337,11 +337,12 @@ class EplbState:
...
@@ -337,11 +337,12 @@ class EplbState:
Args:
Args:
model (MixtureOfExperts): The MoE model.
model (MixtureOfExperts): The MoE model.
is_dummy (bool): If `True`, this is a dummy step and the load
is_dummy (bool): If `True`, this is a dummy step and the load
metrics recorded in this forward pass will not count.
Defaults
metrics recorded in this forward pass will not count.
to `False`.
Defaults
to `False`.
is_profile (bool): If `True`, perform a dummy rearrangement
is_profile (bool): If `True`, perform a dummy rearrangement
with maximum communication cost. This is used in `profile_run`
with maximum communication cost. This is used in
to reserve enough memory for the communication buffer.
`profile_run` to reserve enough memory
for the communication buffer.
log_stats (bool): If `True`, log the expert load metrics.
log_stats (bool): If `True`, log the expert load metrics.
# Stats
# Stats
...
...
vllm/distributed/eplb/rebalance_algo.py
View file @
47f670b0
...
@@ -109,13 +109,16 @@ def rebalance_experts_hierarchical(
...
@@ -109,13 +109,16 @@ def rebalance_experts_hierarchical(
num_physical_experts: number of physical experts after replication
num_physical_experts: number of physical experts after replication
num_groups: number of expert groups
num_groups: number of expert groups
num_nodes: number of server nodes, where the intra-node network
num_nodes: number of server nodes, where the intra-node network
(e.g, NVLink) is faster
(e.g
.
, NVLink) is faster
num_gpus: number of GPUs, must be a multiple of `num_nodes`
num_gpus: number of GPUs, must be a multiple of `num_nodes`
Returns:
Returns:
physical_to_logical_map: [num_moe_layers, num_physical_experts]
physical_to_logical_map (torch.Tensor):
logical_to_physical_map: [num_moe_layers, num_logical_experts, X]
[num_moe_layers, num_physical_experts]
logical_count: [num_moe_layers, num_logical_experts]
logical_to_physical_map (torch.Tensor):
[num_moe_layers, num_logical_experts, X]
logical_count (torch.Tensor):
[num_moe_layers, num_logical_experts]
"""
"""
num_layers
,
num_logical_experts
=
weight
.
shape
num_layers
,
num_logical_experts
=
weight
.
shape
assert
num_logical_experts
%
num_groups
==
0
assert
num_logical_experts
%
num_groups
==
0
...
@@ -197,11 +200,13 @@ def rebalance_experts(
...
@@ -197,11 +200,13 @@ def rebalance_experts(
num_gpus: number of GPUs, must be a multiple of `num_nodes`
num_gpus: number of GPUs, must be a multiple of `num_nodes`
Returns:
Returns:
physical_to_logical_map: [layers, num_replicas], the expert index of
physical_to_logical_map:
each replica
[layers, num_replicas], the expert index of each replica
logical_to_physical_map: [layers, num_logical_experts, X], the replica
logical_to_physical_map:
indices for each expert
[layers, num_logical_experts, X], the replica indices for each
expert_count: [layers, num_logical_experts], number of physical
expert
expert_count:
[layers, num_logical_experts], number of physical
replicas for each logical expert
replicas for each logical expert
"""
"""
num_layers
,
num_logical_experts
=
weight
.
shape
num_layers
,
num_logical_experts
=
weight
.
shape
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment