Commits · 378db1cda125ee8f43f7b157cf0125a88c66627e · OpenDAS / Megatron-LM

19 Apr, 2023 2 commits
- Merge branch 'checkpoint_util_fix' into 'main' · 378db1cd
  Jared Casper authored Apr 19, 2023
```
Some quick fixes to checkpoint_util.

See merge request ADLR/megatron-lm!568
```
  378db1cd
- Some quick fixes to checkpoint_util. · be1a575e
  Jared Casper authored Apr 19, 2023
  
  be1a575e
14 Apr, 2023 1 commit
- Merge branch 'test-update' into 'main' · 8dbd0757
  Jared Casper authored Apr 13, 2023
```
Update tests with new gold numbers

See merge request ADLR/megatron-lm!565
```
  8dbd0757
13 Apr, 2023 1 commit
- Recent commit changed the bias dropout add jitted function which slightly... · 08e62070
  Jared Casper authored Apr 13, 2023
```
Recent commit changed the bias dropout add jitted function which slightly changes numbers, manually tested to be accurate.
```
  08e62070
09 Apr, 2023 1 commit
- Merge branch 'add_helper_fns' into 'main' · ee084835
  Jared Casper authored Apr 08, 2023
```
Add parallel_state helper functions

See merge request ADLR/megatron-lm!564
```
  ee084835
07 Apr, 2023 4 commits
- add needed helper functions · 3ce6a1c2
  Abhinav Khattar authored Apr 07, 2023
```
Signed-off-by: Abhinav Khattar <aklife97@gmail.com>
```
  3ce6a1c2
- Merge branch 'nextlm-fixes' into 'main' · 9c5e287d
  Jared Casper authored Apr 06, 2023
```
Some quick fixes.

See merge request ADLR/megatron-lm!563
```
  9c5e287d
- Some quick fixes. · 45084d46
  Jared Casper authored Apr 06, 2023
  
  45084d46
- Merge branch 'flash_attention_inference' into 'main' · cdd2afdf
  Jared Casper authored Apr 06, 2023
```
Flash Attention inference fix

See merge request ADLR/megatron-lm!562
```
  cdd2afdf
06 Apr, 2023 17 commits
- dropout variable · a7cce320
  Jimmy Zhang authored Apr 06, 2023
  
  a7cce320
- refactor flash attention · 254e8815
  Jimmy Zhang authored Apr 06, 2023
  
  254e8815
- Flash Attention inference fix · f1a50a3c
  Jimmy Zhang authored Mar 03, 2023
  
  f1a50a3c
- Merge branch 'nextlm-merge-sentencepiece' into 'main' · 717c5274
  Jared Casper authored Apr 06, 2023
```
Add support for GPTSentencePieceTokenizer and related fixes.

See merge request ADLR/megatron-lm!561
```
  717c5274
- Merge branch 'untie_embeddings' into 'main' · 018391a6
  Jared Casper authored Apr 06, 2023
```
Untie Embeddings

See merge request ADLR/megatron-lm!558
```
  018391a6
- Missed some changes from next-best-lm branch. · 3207c19a
  Jared Casper authored Apr 06, 2023
  
  3207c19a
- Add support for GPTSentencePieceTokenizer and related fixes. · 46ffb75c
  Jared Casper authored Apr 06, 2023
  
  46ffb75c
- Merge branch 'next-best-lm/merge-layernorm-1p-main' into 'main' · 4e891fe9
  Jared Casper authored Apr 05, 2023
```
layernorm1p added

See merge request ADLR/megatron-lm!557
```
  4e891fe9
- Merge branch 'next-best-lm/merge-rope-main' into 'main' · 7bd25e26
  Jared Casper authored Apr 05, 2023
```
merging rope to main

See merge request ADLR/megatron-lm!556
```
  7bd25e26
- Add assert in BertModel that untie_embeddings_and_output_weights is False · 1e2a0405
  Jared Casper authored Apr 05, 2023
  
  1e2a0405
- Merge branch 'main' into untie_embeddings · c43d19ab
  Jared Casper authored Apr 05, 2023
  
  c43d19ab
- Merge branch 'main' into next-best-lm/merge-rope-main · 74bd02ec
  Jared Casper authored Apr 05, 2023
  
  74bd02ec
- Merge branch 'main' into next-best-lm/merge-layernorm-1p-main · 33a58153
  Jared Casper authored Apr 05, 2023
  
  33a58153
- Expand on apply-layernorm-1p description a bit. · 05b808ef
  Jared Casper authored Apr 05, 2023
  
  05b808ef
- A bit of cleanup. · 93bed794
  Jared Casper authored Apr 05, 2023
  
  93bed794
- Merge branch 'activations-bias' into 'main' · f6d36d03
  Jared Casper authored Apr 05, 2023
```
Add swiglu and squared relu activations and ability to disable bias.

See merge request ADLR/megatron-lm!553
```
  f6d36d03
- Merge branch 'bugFix' into 'main' · 22f054d8
  Shanmugam Ramasamy authored Apr 05, 2023
```
Update .gitlab-ci.yml

See merge request ADLR/megatron-lm!560
```
  22f054d8
05 Apr, 2023 4 commits
- Update .gitlab-ci.yml · 939383ae
  Shanmugam Ramasamy authored Apr 05, 2023
  
  939383ae
- Fix mpu init · 716da5d8
  MaximumEntropy authored Apr 05, 2023
```
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
```
  716da5d8
- addressing the comments · 5ed304e4
  Mostofa Patwary authored Apr 05, 2023
  
  5ed304e4
- addressing the comments · 0760822b
  Mostofa Patwary authored Apr 05, 2023
  
  0760822b
04 Apr, 2023 5 commits
- Fix output layer init func when untie embeddings is True · 0f6bfdeb
  Sandeep Subramanian authored Feb 06, 2023
  
  0f6bfdeb
- Merge branch 'better-Output' into 'main' · 58020503
  Shanmugam Ramasamy authored Apr 04, 2023
```
Better output

See merge request ADLR/megatron-lm!559
```
  58020503
- Better output · 2221d882
  Shanmugam Ramasamy authored Apr 04, 2023
  
  2221d882
- removed self_attention_pos_emb · e1f1aa06
  Mostofa Patwary authored Apr 04, 2023
  
  e1f1aa06
- addressing comments · bf1da27e
  Mostofa Patwary authored Apr 04, 2023
  
  bf1da27e
03 Apr, 2023 5 commits
- addressing more comments · 5b7736ca
  Mostofa Patwary authored Apr 03, 2023
  
  5b7736ca
- addressing more comments · 07ffa1b7
  Mostofa Patwary authored Apr 03, 2023
  
  07ffa1b7
- remove unneccessary argument check · a9f6569d
  Jimmy Zhang authored Apr 03, 2023
  
  a9f6569d
- Fix spacing · 4fcb2f45
  MaximumEntropy authored Apr 03, 2023
```
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
```
  4fcb2f45
- Merge branch 'untie_embeddings' of... · 1f8bc276
  MaximumEntropy authored Apr 03, 2023
```
Merge branch 'untie_embeddings' of https://gitlab-master.nvidia.com/ADLR/megatron-lm into untie_embeddings
```
  1f8bc276