Commits · 0024a5c66f90c7d3d02f7ef08a773aace6deb155 · OpenDAS / Megatron-LM

19 Apr, 2023 3 commits
- address review comments · 98bb40d9
  Abhinav Khattar authored Apr 19, 2023
```
Signed-off-by: Abhinav Khattar <aklife97@gmail.com>
```
  98bb40d9
- keep previous autocast check · 630811b2
  Abhinav Khattar authored Apr 19, 2023
```
Signed-off-by: Abhinav Khattar <aklife97@gmail.com>
```
  630811b2
- add enable autocast · 513d7d8e
  Abhinav Khattar authored Apr 19, 2023
```
Signed-off-by: Abhinav Khattar <aklife97@gmail.com>
```
  513d7d8e
07 Apr, 2023 1 commit
- add needed helper functions · 3ce6a1c2
  Abhinav Khattar authored Apr 07, 2023
```
Signed-off-by: Abhinav Khattar <aklife97@gmail.com>
```
  3ce6a1c2
03 Apr, 2023 3 commits
- Fix spacing · 4fcb2f45
  MaximumEntropy authored Apr 03, 2023
```
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
```
  4fcb2f45
- Undo parallel state changes · 96f4c5d2
  MaximumEntropy authored Apr 03, 2023
```
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
```
  96f4c5d2
- Initial commit for untied embeddings · 82c7ba57
  MaximumEntropy authored Dec 07, 2022
```
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
```
  82c7ba57
01 Apr, 2023 1 commit
- Fix bug in core pipeline schedule · ff7b28f4
  Jared Casper authored Apr 01, 2023
  
  ff7b28f4
23 Mar, 2023 2 commits
- Move pipeline parallel functionality into core with associated changes. · 3c92fa93
  Jared Casper authored Mar 23, 2023
  
  3c92fa93
- Add contiguous to grad_output in layers.py · 09c2f6e4
  Markel Ausin authored Mar 22, 2023
  
  09c2f6e4
14 Oct, 2022 1 commit
- Remove noop used to try to force scheduling and check for environment variable instead. · bdd97312
  Jared Casper authored Oct 14, 2022
  
  bdd97312
13 Oct, 2022 1 commit
- Adding proper test cases · 8b94a160
  Shanmugam Ramasamy authored Oct 13, 2022
  
  8b94a160
06 Oct, 2022 2 commits
- Adding some basic unit tests · b69e2195
  shanmugamr authored Oct 06, 2022
  
  b69e2195
- Setter for pipeline parallel split rank, remove print · 6defe188
  Eric Harper authored Oct 05, 2022
  
  6defe188
27 Sep, 2022 4 commits
- Support gradient accumulation fusion in fp16. · 07916bf2
  Jared Casper authored Sep 27, 2022
  
  07916bf2
- Error, not warn, if gradient_accumulation_fusion is requested but not available. · 2366716f
  Jared Casper authored Sep 27, 2022
  
  2366716f
- Correct some merge errors. · 55817ec9
  Jared Casper authored Sep 27, 2022
  
  55817ec9
- Small fixes. · 77753d0a
  Jared Casper authored Sep 27, 2022
  
  77753d0a
26 Sep, 2022 2 commits
- Check for pipeline_parallel > 2 when using interleaving. · e7e99721
  Jared Casper authored Sep 26, 2022
  
  e7e99721
- Update exports. · cbf780d0
  Jared Casper authored Sep 26, 2022
  
  cbf780d0
24 Sep, 2022 3 commits
- Improve docstrings, destory global memory buffer. · e00a1ca1
  Jared Casper authored Sep 23, 2022
  
  e00a1ca1
- Move get_num_layers into transformer.py. · a94d0a6e
  Jared Casper authored Sep 23, 2022
  
  a94d0a6e
- Alias core.parallel_state as mpu and use it throughout code. RIP mpu. · 5942af97
  Jared Casper authored Sep 23, 2022
  
  5942af97
23 Sep, 2022 3 commits
- Move layers from mpu to core.tensor_parallel. · c2ea914f
  Jared Casper authored Sep 23, 2022
  
  c2ea914f
- Bring mpu.data into megatron.core. · 209f91c9
  Jared Casper authored Sep 22, 2022
  
  209f91c9
- Start Megatron-Core with vocab parallel cross entropy · 2e6a46e4
  Jared Casper authored Sep 22, 2022
```
Also merged in some changed from apex
```
  2e6a46e4