Commits · 91fcbd689b9da189def7fd3cd59e19438ebed22a · OpenDAS / Megatron-LM

16 Aug, 2021 2 commits
- Merge branch 'torch_dist_env_init' into 'main' · 91fcbd68
  Jared Casper authored Aug 16, 2021
```
changed torch distributed init method from tcp to env

See merge request ADLR/megatron-lm!297
```
  91fcbd68
- changed torch distributed init method from tcp to env · ee4fd509
  Mohammad Shoeybi authored Aug 16, 2021
  
  ee4fd509
13 Aug, 2021 1 commit

Merge branch 'lmcafee/pipe-forward-only' into 'main' · 78a69eaf

Jared Casper authored Aug 13, 2021

removed saving of input/output tensors for forward-only passes of pipeline schedules

See merge request ADLR/megatron-lm!300

78a69eaf

12 Aug, 2021 4 commits
- Merge branch 'lmcafee/copygrad-fix-v2' into 'main' · 75740263
  Jared Casper authored Aug 12, 2021
```
Lmcafee/copygrad fix v2

See merge request ADLR/megatron-lm!299
```
  75740263
- Merge branch 'fused_kernels_simd_fix' into 'main' · 2387ce01
  Jared Casper authored Aug 12, 2021
```
fused kernels simd fix for short sequences

See merge request ADLR/megatron-lm!302
```
  2387ce01
- simd fix for upper traingular softmax kernel · 62424a74
  Vijay Korthikanti authored Aug 12, 2021
  
  62424a74
- fused kernels simd fix for short sequences · 5e22689a
  Vijay Korthikanti authored Aug 12, 2021
  
  5e22689a
11 Aug, 2021 1 commit
- added asserts/checks for local ddp and params_have_main_grad · f597f02e
  Lawrence McAfee authored Aug 11, 2021
  
  f597f02e
10 Aug, 2021 2 commits
- Merge branch 'sc21' into 'main' · e269e200
  Jared Casper authored Aug 10, 2021
```
scripts for sc21

See merge request ADLR/megatron-lm!298
```
  e269e200
- scripts for sc21 · de7dc40f
  Mohammad Shoeybi authored Aug 10, 2021
  
  de7dc40f
09 Aug, 2021 2 commits
- removed saving of input/output tensors for forward-only passes of pipeline schedules · 0865c4dc
  Lawrence McAfee authored Aug 09, 2021
  
  0865c4dc
- Merge branch 'main' of ssh://gitlab-master.nvidia.com:12051/ADLR/megatron-lm... · 21a11ca0
  Lawrence McAfee authored Aug 09, 2021
```
Merge branch 'main' of ssh://gitlab-master.nvidia.com:12051/ADLR/megatron-lm into lmcafee/copygrad-fix-v2
```
  21a11ca0
29 Jul, 2021 2 commits
- Merge branch 'bugfix' into 'main' · 6a680986
  Jared Casper authored Jul 29, 2021
```
Use timers kwargs correctly to prevent bug with new p2p_communication API

See merge request ADLR/megatron-lm!295
```
  6a680986
- Use timers kwargs correctly to prevent bug with new p2p_communication API · e727de99
  Deepak Narayanan authored Jul 29, 2021
  
  e727de99
27 Jul, 2021 2 commits
- Merge branch 'main_p2p' into 'main' · a676bc2d
  Deepak Narayanan authored Jul 27, 2021
```
See merge request ADLR/megatron-lm!293
```
  a676bc2d
- Make it possible to pass in tensor shapes to communication methods in p2p_communication.py · 1dccefd8
  Mostofa Patwary authored Jul 27, 2021
  
  1dccefd8
21 Jul, 2021 2 commits
- Merge branch 'small_refactor' into 'main' · 3db6517a
  Jared Casper authored Jul 21, 2021
```
Use helper method in megatron/schedules.py as intended

See merge request ADLR/megatron-lm!292
```
  3db6517a
- Use helper method in megatron/schedules.py as intended · 77bff386
  Deepak Narayanan authored Jul 21, 2021
  
  77bff386
16 Jul, 2021 1 commit
- (conditionally) release grad/main_grad memory in copy_model_grads_to_main_grads, after copy · 6f2bff5c
  Lawrence McAfee authored Jul 16, 2021
  
  6f2bff5c
14 Jul, 2021 2 commits
- Merge branch 'lmcafee/tbmem-fix' into 'main' · f3be8ead
  Jared Casper authored Jul 14, 2021
```
added memory stats (allocated/reserved) to tensorboard logging

See merge request ADLR/megatron-lm!285
```
  f3be8ead
- Merge branch 'lmcafee/zerograd-fix' into 'main' · 3202f237
  Jared Casper authored Jul 14, 2021
```
fixed zero_grad for fp32_from_float16_groups

See merge request ADLR/megatron-lm!283
```
  3202f237
13 Jul, 2021 1 commit
- fixed help message; removed redundant destination variable · bc5a8e20
  Lawrence McAfee authored Jul 13, 2021
  
  bc5a8e20
12 Jul, 2021 8 commits
- added comment explaining why fp32_from_float16_groups should be zeroed here · 4e64903d
  Lawrence McAfee authored Jul 12, 2021
  
  4e64903d
- switched tensorboard memory logging from opt-out to opt-in · 236a5ec0
  Lawrence McAfee authored Jul 12, 2021
  
  236a5ec0
- Merge branch 'readme_update' into 'main' · c1075275
  Jared Casper authored Jul 12, 2021
```
Update README to have a small note about interleaved schedule

See merge request ADLR/megatron-lm!286
```
  c1075275
- Merge branch 'main-readme' into 'main' · a279d356
  Mohammad Shoeybi authored Jul 12, 2021
```
updated data processing readme

See merge request ADLR/megatron-lm!287
```
  a279d356
- updated data processing readme · 5f2cb263
  Mostofa Patwary authored Jul 12, 2021
  
  5f2cb263
- Address Mohammad's comment · a037a9c4
  Deepak Narayanan authored Jul 12, 2021
  
  a037a9c4
- Update README to have a small note about interleaved schedule · ab9a7979
  Deepak Narayanan authored Jul 12, 2021
  
  ab9a7979
- added memory stats (allocated/reserved) to tensorboard logging · 24b7c3c0
  Lawrence McAfee authored Jul 12, 2021
  
  24b7c3c0
09 Jul, 2021 1 commit
- fixed zero_grad for fp32_from_float16_groups · 4fd6432d
  Lawrence McAfee authored Jul 09, 2021
  
  4fd6432d
11 Jun, 2021 5 commits
- Merge branch 'github-pr' into 'main' · 90e0a0dd
  Jared Casper authored Jun 11, 2021
```
Pull in some GitHub PRs

See merge request ADLR/megatron-lm!282
```
  90e0a0dd
- Merge branch 'patch-1' of https://github.com/stas00/Megatron-LM into github-pr · 4a35d50a
  Jared Casper authored Jun 10, 2021
  
  4a35d50a
- Merge branch 'typo-fix' of https://github.com/devrimcavusoglu/Megatron-LM into github-pr · e1318f0e
  Jared Casper authored Jun 10, 2021
  
  e1318f0e
- Merge branch 't5' of https://github.com/stas00/Megatron-LM into github-pr · 7898c9a4
  Jared Casper authored Jun 10, 2021
  
  7898c9a4
- Merge branch 'main_retriver_merge_dpr' into 'main' · 82b69e86
  Jared Casper authored Jun 11, 2021
```
Changes in Readme (Retriever)

See merge request ADLR/megatron-lm!281
```
  82b69e86
10 Jun, 2021 4 commits
- updated readme · 4c92ca82
  Mostofa Patwary authored Jun 10, 2021
  
  4c92ca82
- updated readme · 32da2e78
  Mostofa Patwary authored Jun 10, 2021
  
  32da2e78
- updated readme · baf2e2a4
  Mostofa Patwary authored Jun 10, 2021
  
  baf2e2a4
- updated readme · 9d350c9c
  Mostofa Patwary authored Jun 10, 2021
  
  9d350c9c