Skip to content

GitLab

"lib/bindings/python/vscode:/vscode.git/clone" did not exist on "457db719d199964f9c401c20218cbd69059fc371"
Commit e8cf079b authored by zhuwenwen's avatar zhuwenwen
Browse files

Merge branch 'v0.9.2-dev-ds-wm' into 'v0.9.2-dev-ds' 

优化mori ep

See merge request OpenDAS/vllm!7
parents e0ba23b5 5dcc5cb8
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 179 additions and 186 deletions
vllm/distributed/parallel_state.py
View file @ e8cf079b
...@@ -948,7 +948,7 @@ def init_distributed_environment( ...@@ -948,7 +948,7 @@ def init_distributed_environment(
"Fallback Gloo backend is not available.") "Fallback Gloo backend is not available.")
backend = "gloo" backend = "gloo"
# this backend is used for WORLD # this backend is used for WORLD
parallel_config = config.parallel_config
data_parallel_size = parallel_config.data_parallel_size data_parallel_size = parallel_config.data_parallel_size
use_mori_ep = envs.VLLM_USE_MORI_EP and data_parallel_size > 1 and parallel_config.enable_expert_parallel use_mori_ep = envs.VLLM_USE_MORI_EP and data_parallel_size > 1 and parallel_config.enable_expert_parallel
if use_mori_ep: if use_mori_ep:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment