Commits · eddf759372c118fcc4cf05790b20f2685450baeb · OpenDAS / Megatron-LM

16 Aug, 2021 1 commit

Destroy more groups in `destroy_model_parallel` · eddf7593

eqy authored Aug 16, 2021

Some tests expect a clean model parallel slate and complain if a previous test left something behind; this change clears more variables that the tests complain about.

eddf7593

12 Aug, 2021 3 commits
- Merge branch 'fused_kernels_simd_fix' into 'main' · 2387ce01
  Jared Casper authored Aug 12, 2021
```
fused kernels simd fix for short sequences

See merge request ADLR/megatron-lm!302
```
  2387ce01
- simd fix for upper traingular softmax kernel · 62424a74
  Vijay Korthikanti authored Aug 12, 2021
  
  62424a74
- fused kernels simd fix for short sequences · 5e22689a
  Vijay Korthikanti authored Aug 12, 2021
  
  5e22689a
10 Aug, 2021 2 commits
- Merge branch 'sc21' into 'main' · e269e200
  Jared Casper authored Aug 10, 2021
```
scripts for sc21

See merge request ADLR/megatron-lm!298
```
  e269e200
- scripts for sc21 · de7dc40f
  Mohammad Shoeybi authored Aug 10, 2021
  
  de7dc40f
29 Jul, 2021 2 commits
- Merge branch 'bugfix' into 'main' · 6a680986
  Jared Casper authored Jul 29, 2021
```
Use timers kwargs correctly to prevent bug with new p2p_communication API

See merge request ADLR/megatron-lm!295
```
  6a680986
- Use timers kwargs correctly to prevent bug with new p2p_communication API · e727de99
  Deepak Narayanan authored Jul 29, 2021
  
  e727de99
27 Jul, 2021 2 commits
- Merge branch 'main_p2p' into 'main' · a676bc2d
  Deepak Narayanan authored Jul 27, 2021
```
See merge request ADLR/megatron-lm!293
```
  a676bc2d
- Make it possible to pass in tensor shapes to communication methods in p2p_communication.py · 1dccefd8
  Mostofa Patwary authored Jul 27, 2021
  
  1dccefd8
21 Jul, 2021 2 commits
- Merge branch 'small_refactor' into 'main' · 3db6517a
  Jared Casper authored Jul 21, 2021
```
Use helper method in megatron/schedules.py as intended

See merge request ADLR/megatron-lm!292
```
  3db6517a
- Use helper method in megatron/schedules.py as intended · 77bff386
  Deepak Narayanan authored Jul 21, 2021
  
  77bff386
14 Jul, 2021 2 commits
- Merge branch 'lmcafee/tbmem-fix' into 'main' · f3be8ead
  Jared Casper authored Jul 14, 2021
```
added memory stats (allocated/reserved) to tensorboard logging

See merge request ADLR/megatron-lm!285
```
  f3be8ead
- Merge branch 'lmcafee/zerograd-fix' into 'main' · 3202f237
  Jared Casper authored Jul 14, 2021
```
fixed zero_grad for fp32_from_float16_groups

See merge request ADLR/megatron-lm!283
```
  3202f237
13 Jul, 2021 1 commit
- fixed help message; removed redundant destination variable · bc5a8e20
  Lawrence McAfee authored Jul 13, 2021
  
  bc5a8e20
12 Jul, 2021 8 commits
- added comment explaining why fp32_from_float16_groups should be zeroed here · 4e64903d
  Lawrence McAfee authored Jul 12, 2021
  
  4e64903d
- switched tensorboard memory logging from opt-out to opt-in · 236a5ec0
  Lawrence McAfee authored Jul 12, 2021
  
  236a5ec0
- Merge branch 'readme_update' into 'main' · c1075275
  Jared Casper authored Jul 12, 2021
```
Update README to have a small note about interleaved schedule

See merge request ADLR/megatron-lm!286
```
  c1075275
- Merge branch 'main-readme' into 'main' · a279d356
  Mohammad Shoeybi authored Jul 12, 2021
```
updated data processing readme

See merge request ADLR/megatron-lm!287
```
  a279d356
- updated data processing readme · 5f2cb263
  Mostofa Patwary authored Jul 12, 2021
  
  5f2cb263
- Address Mohammad's comment · a037a9c4
  Deepak Narayanan authored Jul 12, 2021
  
  a037a9c4
- Update README to have a small note about interleaved schedule · ab9a7979
  Deepak Narayanan authored Jul 12, 2021
  
  ab9a7979
- added memory stats (allocated/reserved) to tensorboard logging · 24b7c3c0
  Lawrence McAfee authored Jul 12, 2021
  
  24b7c3c0
09 Jul, 2021 1 commit
- fixed zero_grad for fp32_from_float16_groups · 4fd6432d
  Lawrence McAfee authored Jul 09, 2021
  
  4fd6432d
11 Jun, 2021 5 commits
- Merge branch 'github-pr' into 'main' · 90e0a0dd
  Jared Casper authored Jun 11, 2021
```
Pull in some GitHub PRs

See merge request ADLR/megatron-lm!282
```
  90e0a0dd
- Merge branch 'patch-1' of https://github.com/stas00/Megatron-LM into github-pr · 4a35d50a
  Jared Casper authored Jun 10, 2021
  
  4a35d50a
- Merge branch 'typo-fix' of https://github.com/devrimcavusoglu/Megatron-LM into github-pr · e1318f0e
  Jared Casper authored Jun 10, 2021
  
  e1318f0e
- Merge branch 't5' of https://github.com/stas00/Megatron-LM into github-pr · 7898c9a4
  Jared Casper authored Jun 10, 2021
  
  7898c9a4
- Merge branch 'main_retriver_merge_dpr' into 'main' · 82b69e86
  Jared Casper authored Jun 11, 2021
```
Changes in Readme (Retriever)

See merge request ADLR/megatron-lm!281
```
  82b69e86
10 Jun, 2021 11 commits
- updated readme · 4c92ca82
  Mostofa Patwary authored Jun 10, 2021
  
  4c92ca82
- updated readme · 32da2e78
  Mostofa Patwary authored Jun 10, 2021
  
  32da2e78
- updated readme · baf2e2a4
  Mostofa Patwary authored Jun 10, 2021
  
  baf2e2a4
- updated readme · 9d350c9c
  Mostofa Patwary authored Jun 10, 2021
  
  9d350c9c
- Merge branch 't5_scripts' into 'main' · 2be1e510
  Jared Casper authored Jun 10, 2021
```
Update T5 scripts

See merge request ADLR/megatron-lm!279
```
  2be1e510
- Merge branch 'main_retriver_merge_dpr' into 'main' · 598d7ee2
  Jared Casper authored Jun 10, 2021
```
DPR evaluation hangs and Readme

See merge request ADLR/megatron-lm!280
```
  598d7ee2
- Merge branch 'main_retriver_merge_dpr' of... · 98113c69
  Mostofa Patwary authored Jun 10, 2021
```
Merge branch 'main_retriver_merge_dpr' of ssh://gitlab-master.nvidia.com:12051/ADLR/megatron-lm into main_retriver_merge_dpr
```
  98113c69
- addressed comments · 28450473
  Mostofa Patwary authored Jun 10, 2021
  
  28450473
- Clean up README.md a bit · 473127f9
  Jared Casper authored Jun 10, 2021
  
  473127f9
- Adding readme · c45109ed
  Mostofa Patwary authored Jun 09, 2021
  
  c45109ed
- Adding readme · e287bf00
  Mostofa Patwary authored Jun 09, 2021
  
  e287bf00