Skip to content

GitLab

Commit ee3ff5df authored by silencealiang's avatar silencealiang
Browse files

Update deepseekv3 parameters to avoid expert select fault

parent 54740897
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
examples/deepseek_v3/train_deepseekv3_671B_1nodes.sh
View file @ ee3ff5df
...@@ -114,8 +114,8 @@ moe_options=" \ ...@@ -114,8 +114,8 @@ moe_options=" \
--moe-pad-expert-input-to-capacity \ --moe-pad-expert-input-to-capacity \
--moe-token-dispatcher-type alltoall \ --moe-token-dispatcher-type alltoall \
--moe-router-topk ${ROUTER_TOPK} \ --moe-router-topk ${ROUTER_TOPK} \
--moe-router-group-topk 2 \ --moe-router-group-topk 1 \
--moe-router-num-groups 4 \ --moe-router-num-groups 1 \
--num-experts ${NUM_EXPERTS} \ --num-experts ${NUM_EXPERTS} \
--expert-model-parallel-size ${EP} \ --expert-model-parallel-size ${EP} \
--expert-tensor-parallel-size ${ETP} \ --expert-tensor-parallel-size ${ETP} \
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment