Commits · 0f6bfdeba673e9e71eee2ad59aff7e0d49f891b8 · OpenDAS / Megatron-LM

04 Apr, 2023 1 commit
- Fix output layer init func when untie embeddings is True · 0f6bfdeb
  Sandeep Subramanian authored Feb 06, 2023
  
  0f6bfdeb
03 Apr, 2023 9 commits
- remove unneccessary argument check · a9f6569d
  Jimmy Zhang authored Apr 03, 2023
  
  a9f6569d
- Fix spacing · 4fcb2f45
  MaximumEntropy authored Apr 03, 2023
```
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
```
  4fcb2f45
- Undo parallel state changes · 96f4c5d2
  MaximumEntropy authored Apr 03, 2023
```
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
```
  96f4c5d2
- assertion check for T5 and untied embeddings · 6b50a8c6
  Jimmy Zhang authored Apr 03, 2023
  
  6b50a8c6
- Save word embeddings for head only only if embeddings are not untied · 18b26ec6
  Sandeep Subramanian authored Feb 15, 2023
  
  18b26ec6
- disable embedding addreduce if untie_embeddings_and_output_weights · f11b4c99
  Jimmy Zhang authored Jan 31, 2023
  
  f11b4c99
- Add assert for pp and untie · a3fbac58
  MaximumEntropy authored Dec 08, 2022
```
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
```
  a3fbac58
- Fix checkpointing · 36c3a1c1
  MaximumEntropy authored Dec 08, 2022
```
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
```
  36c3a1c1
- Initial commit for untied embeddings · 82c7ba57
  MaximumEntropy authored Dec 07, 2022
```
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
```
  82c7ba57
01 Apr, 2023 1 commit
- Fix bug in core pipeline schedule · ff7b28f4
  Jared Casper authored Apr 01, 2023
  
  ff7b28f4
31 Mar, 2023 1 commit
- fix bug in uniform activation recompute · fc2c81d3
  Jared Casper authored Mar 31, 2023
  
  fc2c81d3
23 Mar, 2023 2 commits
- Move pipeline parallel functionality into core with associated changes. · 3c92fa93
  Jared Casper authored Mar 23, 2023
  
  3c92fa93
- Add contiguous to grad_output in layers.py · 09c2f6e4
  Markel Ausin authored Mar 22, 2023
  
  09c2f6e4
22 Feb, 2023 1 commit
- Retro · 17a6044d
  Lawrence McAfee authored Feb 21, 2023
  
  17a6044d
04 Jan, 2023 1 commit
- Remove FA's check for headdim <= 128 · 9200e43a
  Tri Dao authored Jan 03, 2023
  
  9200e43a
22 Dec, 2022 1 commit
- Transformer Engine Integration Rebase · 3499542e
  John Kamalu authored Dec 21, 2022
  
  3499542e
09 Dec, 2022 1 commit
- Integrate FlashAttention into Megatron-LM · d693034e
  Tri Dao authored Dec 08, 2022
  
  d693034e
29 Nov, 2022 1 commit
- bug fixes in partitioned data preprocessor · 84a43b13
  Mike Chrzanowski authored Nov 29, 2022
  
  84a43b13
24 Nov, 2022 1 commit
- Open sourcing lm detoxification code · d48d95ab
  Boxin Wang authored Nov 23, 2022
  
  d48d95ab
22 Nov, 2022 1 commit
- Fix DropPath for hidden shape [s, b, h] · 6d45a903
  Yu Yao authored Nov 22, 2022
  
  6d45a903
17 Nov, 2022 1 commit
- Data Preprocessing Optimizations · 7fc96111
  Vijay Korthikanti authored Nov 17, 2022
  
  7fc96111
11 Nov, 2022 1 commit
- Support for variable sequence lengths across micro-batches · c3e688d3
  Vijay Korthikanti authored Nov 10, 2022
  
  c3e688d3
10 Nov, 2022 1 commit
- ViT Backbone Tensor Shape Fix · fabd3e4e
  Yu Yao authored Nov 10, 2022
  
  fabd3e4e
08 Nov, 2022 1 commit
- Fix merge error. · e0a12fe1
  Jared Casper authored Nov 08, 2022
  
  e0a12fe1
27 Oct, 2022 1 commit
- Sending in prompts with the wrong type hangs the server. This is a check to make sure it's a list · 2fdd54ec
  rprenger authored Oct 27, 2022
  
  2fdd54ec
20 Oct, 2022 1 commit
- Disable newline after colon · 544e2502
  Peng Xu authored Oct 19, 2022
  
  544e2502
14 Oct, 2022 2 commits
- Remove noop used to try to force scheduling and check for environment variable instead. · bdd97312
  Jared Casper authored Oct 14, 2022
  
  bdd97312
- inverse_square_root learning param schedule · dbed5e07
  Vijay Korthikanti authored Oct 14, 2022
  
  dbed5e07
13 Oct, 2022 1 commit
- Adding proper test cases · 8b94a160
  Shanmugam Ramasamy authored Oct 13, 2022
  
  8b94a160
11 Oct, 2022 1 commit
- address review comments · 69f3249f
  Vijay Korthikanti authored Oct 11, 2022
  
  69f3249f
10 Oct, 2022 1 commit
- removed assert for the case of evaluation only without training · 45cd4e0b
  Anmol Gupta authored Oct 10, 2022
  
  45cd4e0b
06 Oct, 2022 2 commits
- Adding some basic unit tests · b69e2195
  shanmugamr authored Oct 06, 2022
  
  b69e2195
- Setter for pipeline parallel split rank, remove print · 6defe188
  Eric Harper authored Oct 05, 2022
  
  6defe188
05 Oct, 2022 2 commits
- fixed the timer issue for the case with no pipelining · 2514892c
  mshoeybi authored Oct 05, 2022
  
  2514892c
- support for separate dataset files for train, valid and test · 63e59949
  Anmol Gupta authored Oct 05, 2022
  
  63e59949
04 Oct, 2022 1 commit
- different encoder/decoder num-layers support · cbf8250b
  Vijay Korthikanti authored Oct 03, 2022
  
  cbf8250b
30 Sep, 2022 1 commit
- Perform distributed optimizer's all-gather in param dtype (instead of grad dtype) · 57bfa7ca
  Lawrence McAfee authored Sep 30, 2022
  
  57bfa7ca
27 Sep, 2022 2 commits
- Support gradient accumulation fusion in fp16. · 07916bf2
  Jared Casper authored Sep 27, 2022
  
  07916bf2
- Error, not warn, if gradient_accumulation_fusion is requested but not available. · 2366716f
  Jared Casper authored Sep 27, 2022
  
  2366716f