megatron/optimizer/optimizer.py · c9a595549c6dbf8338d4b13dab4a758ccdcb110c · wxj / Megatron-LM · GitLab

"megatron/legacy/mpu/tests/test_random.py" did not exist on "1afe3541ecd4327070471684daa454c9fb1a1006"

Find file Blame History Permalink

Merge branch 'main' into lmcafee/distrib-opt-nodupe · c9a59554
Lawrence McAfee authored May 24, 2022
```
merge sequence parallelism's layernorm all-reduce into distributed optimizer.
```
c9a59554

optimizer.py 28.8 KB