Commits · 46ffb75c4ba15f7ce5266bcb5f2eb03376ea2301 · OpenDAS / Megatron-LM

06 Apr, 2023 3 commits
- Add support for GPTSentencePieceTokenizer and related fixes. · 46ffb75c
  Jared Casper authored Apr 06, 2023
  
  46ffb75c
- Expand on apply-layernorm-1p description a bit. · 05b808ef
  Jared Casper authored Apr 05, 2023
  
  05b808ef
- A bit of cleanup. · 93bed794
  Jared Casper authored Apr 05, 2023
  
  93bed794
05 Apr, 2023 2 commits
- addressing the comments · 5ed304e4
  Mostofa Patwary authored Apr 05, 2023
  
  5ed304e4
- addressing the comments · 0760822b
  Mostofa Patwary authored Apr 05, 2023
  
  0760822b
04 Apr, 2023 2 commits
- removed self_attention_pos_emb · e1f1aa06
  Mostofa Patwary authored Apr 04, 2023
  
  e1f1aa06
- addressing comments · bf1da27e
  Mostofa Patwary authored Apr 04, 2023
  
  bf1da27e
03 Apr, 2023 4 commits
- addressing more comments · 5b7736ca
  Mostofa Patwary authored Apr 03, 2023
  
  5b7736ca
- addressing more comments · 07ffa1b7
  Mostofa Patwary authored Apr 03, 2023
  
  07ffa1b7
- addressed the comments · df521589
  Mostofa Patwary authored Apr 03, 2023
  
  df521589
- Addressing comments. · 420eec74
  Jared Casper authored Apr 03, 2023
  
  420eec74
02 Apr, 2023 1 commit
- layernorm1p added · 5e079c87
  Mostofa Patwary authored Apr 02, 2023
  
  5e079c87
01 Apr, 2023 2 commits
- Add swiglu and squared relu activations and ability to disable bias. · 2cc3dac7
  Jared Casper authored Apr 01, 2023
  
  2cc3dac7
- Fix bug in core pipeline schedule · ff7b28f4
  Jared Casper authored Apr 01, 2023
  
  ff7b28f4
31 Mar, 2023 2 commits
- merging rope to main · 1e0e555c
  Mostofa Patwary authored Mar 31, 2023
  
  1e0e555c
- fix bug in uniform activation recompute · fc2c81d3
  Jared Casper authored Mar 31, 2023
  
  fc2c81d3
23 Mar, 2023 2 commits
- Move pipeline parallel functionality into core with associated changes. · 3c92fa93
  Jared Casper authored Mar 23, 2023
  
  3c92fa93
- Add contiguous to grad_output in layers.py · 09c2f6e4
  Markel Ausin authored Mar 22, 2023
  
  09c2f6e4
22 Feb, 2023 1 commit
- Retro · 17a6044d
  Lawrence McAfee authored Feb 21, 2023
  
  17a6044d
04 Jan, 2023 1 commit
- Remove FA's check for headdim <= 128 · 9200e43a
  Tri Dao authored Jan 03, 2023
  
  9200e43a
22 Dec, 2022 1 commit
- Transformer Engine Integration Rebase · 3499542e
  John Kamalu authored Dec 21, 2022
  
  3499542e
09 Dec, 2022 1 commit
- Integrate FlashAttention into Megatron-LM · d693034e
  Tri Dao authored Dec 08, 2022
  
  d693034e
29 Nov, 2022 1 commit
- bug fixes in partitioned data preprocessor · 84a43b13
  Mike Chrzanowski authored Nov 29, 2022
  
  84a43b13
24 Nov, 2022 1 commit
- Open sourcing lm detoxification code · d48d95ab
  Boxin Wang authored Nov 23, 2022
  
  d48d95ab
22 Nov, 2022 1 commit
- Fix DropPath for hidden shape [s, b, h] · 6d45a903
  Yu Yao authored Nov 22, 2022
  
  6d45a903
17 Nov, 2022 1 commit
- Data Preprocessing Optimizations · 7fc96111
  Vijay Korthikanti authored Nov 17, 2022
  
  7fc96111
11 Nov, 2022 1 commit
- Support for variable sequence lengths across micro-batches · c3e688d3
  Vijay Korthikanti authored Nov 10, 2022
  
  c3e688d3
10 Nov, 2022 1 commit
- ViT Backbone Tensor Shape Fix · fabd3e4e
  Yu Yao authored Nov 10, 2022
  
  fabd3e4e
08 Nov, 2022 1 commit
- Fix merge error. · e0a12fe1
  Jared Casper authored Nov 08, 2022
  
  e0a12fe1
27 Oct, 2022 1 commit
- Sending in prompts with the wrong type hangs the server. This is a check to make sure it's a list · 2fdd54ec
  rprenger authored Oct 27, 2022
  
  2fdd54ec
20 Oct, 2022 1 commit
- Disable newline after colon · 544e2502
  Peng Xu authored Oct 19, 2022
  
  544e2502
14 Oct, 2022 2 commits
- Remove noop used to try to force scheduling and check for environment variable instead. · bdd97312
  Jared Casper authored Oct 14, 2022
  
  bdd97312
- inverse_square_root learning param schedule · dbed5e07
  Vijay Korthikanti authored Oct 14, 2022
  
  dbed5e07
13 Oct, 2022 1 commit
- Adding proper test cases · 8b94a160
  Shanmugam Ramasamy authored Oct 13, 2022
  
  8b94a160
11 Oct, 2022 1 commit
- address review comments · 69f3249f
  Vijay Korthikanti authored Oct 11, 2022
  
  69f3249f
10 Oct, 2022 1 commit
- removed assert for the case of evaluation only without training · 45cd4e0b
  Anmol Gupta authored Oct 10, 2022
  
  45cd4e0b
06 Oct, 2022 2 commits
- Adding some basic unit tests · b69e2195
  shanmugamr authored Oct 06, 2022
  
  b69e2195
- Setter for pipeline parallel split rank, remove print · 6defe188
  Eric Harper authored Oct 05, 2022
  
  6defe188
05 Oct, 2022 2 commits
- fixed the timer issue for the case with no pipelining · 2514892c
  mshoeybi authored Oct 05, 2022
  
  2514892c
- support for separate dataset files for train, valid and test · 63e59949
  Anmol Gupta authored Oct 05, 2022
  
  63e59949