Commits · e270f68af783a99c90a09c06506a8c2b8ee6f2c9 · OpenDAS / Megatron-LM

02 Apr, 2021 1 commit
- Addressed MR comments, mostly adding comments to code. · e270f68a
  Jared Casper authored Apr 02, 2021
  
  e270f68a
24 Mar, 2021 3 commits
- Fixing text generation and zeroshot eval and addressing comments. · 64a83fb5
  Jared Casper authored Mar 24, 2021
  
  64a83fb5
- Tasks seems to be working. · b938ec51
  Jared Casper authored Mar 23, 2021
  
  b938ec51
- pipeline code simplification · 3b91262e
  Vijay Korthikanti authored Mar 02, 2021
  
  3b91262e
20 Mar, 2021 1 commit
- Make sure pipeline-model-parallel size is greater than 2 for interleaved schedule · 182841f7
  Deepak Narayanan authored Mar 20, 2021
  
  182841f7
19 Mar, 2021 2 commits
- ICT zeroshot evaluation · fcfd0949
  Mostofa Patwary authored Mar 19, 2021
  
  fcfd0949
- Bfloat fused softmax + fused layer norm · 0fa7175f
  Mohammad Shoeybi authored Mar 19, 2021
  
  0fa7175f
18 Mar, 2021 1 commit
- refactored the fused kernels build · 0d5188c1
  Mohammad Shoeybi authored Mar 17, 2021
  
  0d5188c1
17 Mar, 2021 1 commit
- fixes to upper triangular masked softmax fusion kernel · 3b12ab15
  Vijay Korthikanti authored Mar 16, 2021
  
  3b12ab15
16 Mar, 2021 2 commits
- minor fixes · 531152d9
  Vijay Korthikanti authored Mar 16, 2021
  
  531152d9
- softmax data load/store optimization · b1a83375
  Vijay Korthikanti authored Mar 16, 2021
  
  b1a83375
08 Mar, 2021 1 commit
- Bfloat with fp32 grad acc · b4bc51b1
  Mohammad Shoeybi authored Mar 08, 2021
  
  b4bc51b1
04 Mar, 2021 3 commits
- Fix syntax · 57437cb1
  Rewon Child authored Mar 04, 2021
  
  57437cb1
- Update argument names and fix merge error · 0aff3629
  Rewon Child authored Mar 04, 2021
  
  0aff3629
- Update comments · 8676baca
  Rewon Child authored Mar 04, 2021
  
  8676baca
03 Mar, 2021 3 commits
- Remove pipeline stall timing (and avoid Explorer perf regressions) · ab09d819
  Deepak Narayanan authored Mar 03, 2021
  
  ab09d819
- Bring back call to ring_exchange() in _communicate() · cbad126a
  Deepak Narayanan authored Mar 03, 2021
  
  cbad126a
- Get PyTorch batched communication API working for interleaved schedule · 78cf869f
  Deepak Narayanan authored Feb 25, 2021
  
  78cf869f
26 Feb, 2021 2 commits
- Fix deadlock when get_num_microbatches() < pipeline-parallel size (don't try... · 08ddde0c
  Deepak Narayanan authored Feb 26, 2021
```
Fix deadlock when get_num_microbatches() < pipeline-parallel size (don't try to measure pipeline stall)
```
  08ddde0c
- Support Torch DDP for single-stage, num_microbatches() > 1 · 65eeb427
  Deepak Narayanan authored Feb 24, 2021
  
  65eeb427
25 Feb, 2021 1 commit
- Don't import deprecated model from realm_model which is broken. · c6f18ccf
  Jared Casper authored Feb 25, 2021
  
  c6f18ccf
23 Feb, 2021 3 commits
- Add option to log zeros · 4e77e7c6
  Rewon Child authored Feb 23, 2021
  
  4e77e7c6
- Added code for building embeddings and savings · bcd605f8
  Mostofa Patwary authored Feb 23, 2021
  
  bcd605f8
- fix warning condition · e055f3cf
  Vijay Korthikanti authored Feb 22, 2021
  
  e055f3cf
22 Feb, 2021 1 commit
- Fix interleaved schedule assertion · b7067cbd
  Deepak Narayanan authored Feb 22, 2021
  
  b7067cbd
18 Feb, 2021 4 commits
- addressed the comments · 3f3ba5e7
  Mostofa Patwary authored Feb 18, 2021
  
  3f3ba5e7
- fixed an issue with ICT · 3d96729a
  Mostofa Patwary authored Feb 17, 2021
  
  3d96729a
- fixed a minor bug · 816fb890
  Mostofa Patwary authored Feb 17, 2021
  
  816fb890
- evaluation code ongoing · 612f438a
  Mostofa Patwary authored Feb 17, 2021
  
  612f438a
17 Feb, 2021 1 commit
- Turn scatter-gather optimization on by default · 72e4230b
  Deepak Narayanan authored Feb 16, 2021
  
  72e4230b
15 Feb, 2021 2 commits
- addressed the comments given by Mohammad · 447c1171
  Mostofa Patwary authored Feb 15, 2021
  
  447c1171
- minor change · 360fb6b7
  Mostofa Patwary authored Feb 15, 2021
  
  360fb6b7
13 Feb, 2021 2 commits
- Change argument to control the number of model chunks in a stage · dcef9069
  Deepak Narayanan authored Feb 13, 2021
  
  dcef9069
- More comments and some cleanup (e.g., better variable names) · 5489bda9
  Deepak Narayanan authored Feb 13, 2021
  
  5489bda9
10 Feb, 2021 4 commits
- Comments in megatron/schedules.py and address a few more comments · 626645c0
  Deepak Narayanan authored Feb 10, 2021
  
  626645c0
- removed the function compile_helper · 22a3d81a
  Mostofa Patwary authored Feb 09, 2021
  
  22a3d81a
- started evaluation of ICT · 613d0fe8
  Mostofa Patwary authored Feb 09, 2021
  
  613d0fe8
- Move unwrap to megatron/utils.py and clean up imports in megatron/schedules.py · cc691cbf
  Deepak Narayanan authored Feb 10, 2021
  
  cc691cbf
09 Feb, 2021 2 commits
- Compute tensor chunk size more cleanly, and add assertion for global batch size · e3e5ea89
  Deepak Narayanan authored Jan 20, 2021
  
  e3e5ea89
- Break up tensors sent between pipeline stages into smaller chunks that can be all-gathered · 27fc4689
  Deepak Narayanan authored Jan 20, 2021
  
  27fc4689