Commits · 77753d0a05f07a0d49afd3dd97afd00e0b04885a · OpenDAS / Megatron-LM

27 Sep, 2022 1 commit
- Small fixes. · 77753d0a
  Jared Casper authored Sep 27, 2022
  
  77753d0a
26 Sep, 2022 2 commits
- Check for pipeline_parallel > 2 when using interleaving. · e7e99721
  Jared Casper authored Sep 26, 2022
  
  e7e99721
- Update exports. · cbf780d0
  Jared Casper authored Sep 26, 2022
  
  cbf780d0
24 Sep, 2022 3 commits
- Improve docstrings, destory global memory buffer. · e00a1ca1
  Jared Casper authored Sep 23, 2022
  
  e00a1ca1
- Move get_num_layers into transformer.py. · a94d0a6e
  Jared Casper authored Sep 23, 2022
  
  a94d0a6e
- Alias core.parallel_state as mpu and use it throughout code. RIP mpu. · 5942af97
  Jared Casper authored Sep 23, 2022
  
  5942af97
23 Sep, 2022 3 commits
- Move layers from mpu to core.tensor_parallel. · c2ea914f
  Jared Casper authored Sep 23, 2022
  
  c2ea914f
- Bring mpu.data into megatron.core. · 209f91c9
  Jared Casper authored Sep 22, 2022
  
  209f91c9
- Start Megatron-Core with vocab parallel cross entropy · 2e6a46e4
  Jared Casper authored Sep 22, 2022
```
Also merged in some changed from apex
```
  2e6a46e4