Commits · da1c96e994a91c6e863b4cb9c70e7628ebd74f13 · OpenDAS / Megatron-LM

"integration-tests/models/test_flash_mistral.py" did not exist on "d5b5bc750fd8d00920cb4cb5e5abe121457d717f"

30 Jul, 2021 3 commits
- Add assertion for now preventing usage of pipeline parallelism with T5 model · da1c96e9
  Deepak Narayanan authored Jul 30, 2021
  
  da1c96e9
- Comment in pretrain_t5.py to explain how pipeline parallelism is implemented for T5 model · 2babcaf6
  Deepak Narayanan authored Jul 30, 2021
  
  2babcaf6
- Support for pipeline parallelism in T5 model · 46c74b4c
  Deepak Narayanan authored Jun 22, 2021
```
- Accumulate encoder hidden state gradient to handle skip connection
- Correctly compute the number of layers in encoder / decoder for T5 model
- Ensure e weights are initialized the same way in embeddings
- Synchronize embedding gradients across encoder and decoder for T5 model
- Support for checkpoint loading and saving
```
  46c74b4c
29 Jul, 2021 2 commits
- Merge branch 'bugfix' into 'main' · 6a680986
  Jared Casper authored Jul 29, 2021
```
Use timers kwargs correctly to prevent bug with new p2p_communication API

See merge request ADLR/megatron-lm!295
```
  6a680986
- Use timers kwargs correctly to prevent bug with new p2p_communication API · e727de99
  Deepak Narayanan authored Jul 29, 2021
  
  e727de99
27 Jul, 2021 2 commits
- Merge branch 'main_p2p' into 'main' · a676bc2d
  Deepak Narayanan authored Jul 27, 2021
```
See merge request ADLR/megatron-lm!293
```
  a676bc2d
- Make it possible to pass in tensor shapes to communication methods in p2p_communication.py · 1dccefd8
  Mostofa Patwary authored Jul 27, 2021
  
  1dccefd8
21 Jul, 2021 2 commits
- Merge branch 'small_refactor' into 'main' · 3db6517a
  Jared Casper authored Jul 21, 2021
```
Use helper method in megatron/schedules.py as intended

See merge request ADLR/megatron-lm!292
```
  3db6517a
- Use helper method in megatron/schedules.py as intended · 77bff386
  Deepak Narayanan authored Jul 21, 2021
  
  77bff386
14 Jul, 2021 2 commits
- Merge branch 'lmcafee/tbmem-fix' into 'main' · f3be8ead
  Jared Casper authored Jul 14, 2021
```
added memory stats (allocated/reserved) to tensorboard logging

See merge request ADLR/megatron-lm!285
```
  f3be8ead
- Merge branch 'lmcafee/zerograd-fix' into 'main' · 3202f237
  Jared Casper authored Jul 14, 2021
```
fixed zero_grad for fp32_from_float16_groups

See merge request ADLR/megatron-lm!283
```
  3202f237
13 Jul, 2021 1 commit
- fixed help message; removed redundant destination variable · bc5a8e20
  Lawrence McAfee authored Jul 13, 2021
  
  bc5a8e20
12 Jul, 2021 8 commits
- added comment explaining why fp32_from_float16_groups should be zeroed here · 4e64903d
  Lawrence McAfee authored Jul 12, 2021
  
  4e64903d
- switched tensorboard memory logging from opt-out to opt-in · 236a5ec0
  Lawrence McAfee authored Jul 12, 2021
  
  236a5ec0
- Merge branch 'readme_update' into 'main' · c1075275
  Jared Casper authored Jul 12, 2021
```
Update README to have a small note about interleaved schedule

See merge request ADLR/megatron-lm!286
```
  c1075275
- Merge branch 'main-readme' into 'main' · a279d356
  Mohammad Shoeybi authored Jul 12, 2021
```
updated data processing readme

See merge request ADLR/megatron-lm!287
```
  a279d356
- updated data processing readme · 5f2cb263
  Mostofa Patwary authored Jul 12, 2021
  
  5f2cb263
- Address Mohammad's comment · a037a9c4
  Deepak Narayanan authored Jul 12, 2021
  
  a037a9c4
- Update README to have a small note about interleaved schedule · ab9a7979
  Deepak Narayanan authored Jul 12, 2021
  
  ab9a7979
- added memory stats (allocated/reserved) to tensorboard logging · 24b7c3c0
  Lawrence McAfee authored Jul 12, 2021
  
  24b7c3c0
09 Jul, 2021 1 commit
- fixed zero_grad for fp32_from_float16_groups · 4fd6432d
  Lawrence McAfee authored Jul 09, 2021
  
  4fd6432d
11 Jun, 2021 5 commits
- Merge branch 'github-pr' into 'main' · 90e0a0dd
  Jared Casper authored Jun 11, 2021
```
Pull in some GitHub PRs

See merge request ADLR/megatron-lm!282
```
  90e0a0dd
- Merge branch 'patch-1' of https://github.com/stas00/Megatron-LM into github-pr · 4a35d50a
  Jared Casper authored Jun 10, 2021
  
  4a35d50a
- Merge branch 'typo-fix' of https://github.com/devrimcavusoglu/Megatron-LM into github-pr · e1318f0e
  Jared Casper authored Jun 10, 2021
  
  e1318f0e
- Merge branch 't5' of https://github.com/stas00/Megatron-LM into github-pr · 7898c9a4
  Jared Casper authored Jun 10, 2021
  
  7898c9a4
- Merge branch 'main_retriver_merge_dpr' into 'main' · 82b69e86
  Jared Casper authored Jun 11, 2021
```
Changes in Readme (Retriever)

See merge request ADLR/megatron-lm!281
```
  82b69e86
10 Jun, 2021 14 commits
- updated readme · 4c92ca82
  Mostofa Patwary authored Jun 10, 2021
  
  4c92ca82
- updated readme · 32da2e78
  Mostofa Patwary authored Jun 10, 2021
  
  32da2e78
- updated readme · baf2e2a4
  Mostofa Patwary authored Jun 10, 2021
  
  baf2e2a4
- updated readme · 9d350c9c
  Mostofa Patwary authored Jun 10, 2021
  
  9d350c9c
- Merge branch 't5_scripts' into 'main' · 2be1e510
  Jared Casper authored Jun 10, 2021
```
Update T5 scripts

See merge request ADLR/megatron-lm!279
```
  2be1e510
- Merge branch 'main_retriver_merge_dpr' into 'main' · 598d7ee2
  Jared Casper authored Jun 10, 2021
```
DPR evaluation hangs and Readme

See merge request ADLR/megatron-lm!280
```
  598d7ee2
- Merge branch 'main_retriver_merge_dpr' of... · 98113c69
  Mostofa Patwary authored Jun 10, 2021
```
Merge branch 'main_retriver_merge_dpr' of ssh://gitlab-master.nvidia.com:12051/ADLR/megatron-lm into main_retriver_merge_dpr
```
  98113c69
- addressed comments · 28450473
  Mostofa Patwary authored Jun 10, 2021
  
  28450473
- Clean up README.md a bit · 473127f9
  Jared Casper authored Jun 10, 2021
  
  473127f9
- Adding readme · c45109ed
  Mostofa Patwary authored Jun 09, 2021
  
  c45109ed
- Adding readme · e287bf00
  Mostofa Patwary authored Jun 09, 2021
  
  e287bf00
- Adding readme · 293554aa
  Mostofa Patwary authored Jun 09, 2021
  
  293554aa
- Adding readme · 8661ca26
  Mostofa Patwary authored Jun 09, 2021
  
  8661ca26
- Adding readme · bab5cc4e
  Mostofa Patwary authored Jun 09, 2021
  
  bab5cc4e