Commits · b9fcb7b48357bfdd63d1ce617443642d3b697c28 · OpenDAS / Megatron-LM

23 Apr, 2021 1 commit
- implementation dpr · 06076c7a
  Mostofa Patwary authored Apr 23, 2021
  
  06076c7a
21 Apr, 2021 3 commits
- addressed comments · e5ec27d7
  Mostofa Patwary authored Apr 21, 2021
  
  e5ec27d7
- addressed comments · 5a6431f5
  Mostofa Patwary authored Apr 21, 2021
  
  5a6431f5
- addressed comments · 5c2ce593
  Mostofa Patwary authored Apr 21, 2021
  
  5c2ce593
19 Apr, 2021 1 commit
- modified the params · 44bfcb32
  Mostofa Patwary authored Apr 19, 2021
  
  44bfcb32
16 Apr, 2021 2 commits
- added this function for evaluation · 045959cb
  Mostofa Patwary authored Apr 16, 2021
  
  045959cb
- Integrate code from t5_main into existing code. · 48a5e0da
  Jared Casper authored Apr 15, 2021
  
  48a5e0da
09 Apr, 2021 1 commit
- implementing DPR · 10ff0607
  Mostofa Patwary authored Apr 09, 2021
  
  10ff0607
08 Apr, 2021 1 commit
- Release fixes · 50a4b5fa
  Mohammad Shoeybi authored Apr 08, 2021
  
  50a4b5fa
03 Apr, 2021 1 commit
- Small bugfix to make sure refactored code works with interleaved schedule · 6fd78189
  Deepak Narayanan authored Apr 03, 2021
  
  6fd78189
02 Apr, 2021 1 commit
- Addressed MR comments, mostly adding comments to code. · e270f68a
  Jared Casper authored Apr 02, 2021
  
  e270f68a
31 Mar, 2021 1 commit
- removed the checks for bfloat jitting · d28716e8
  mshoeybi authored Mar 30, 2021
  
  d28716e8
24 Mar, 2021 3 commits
- Fixing text generation and zeroshot eval and addressing comments. · 64a83fb5
  Jared Casper authored Mar 24, 2021
  
  64a83fb5
- Tasks seems to be working. · b938ec51
  Jared Casper authored Mar 23, 2021
  
  b938ec51
- pipeline code simplification · 3b91262e
  Vijay Korthikanti authored Mar 02, 2021
  
  3b91262e
20 Mar, 2021 1 commit
- Make sure pipeline-model-parallel size is greater than 2 for interleaved schedule · 182841f7
  Deepak Narayanan authored Mar 20, 2021
  
  182841f7
19 Mar, 2021 3 commits
- ICT zeroshot evaluation · fcfd0949
  Mostofa Patwary authored Mar 19, 2021
  
  fcfd0949
- Bfloat fused softmax + fused layer norm · 0fa7175f
  Mohammad Shoeybi authored Mar 19, 2021
  
  0fa7175f
- Fixed based on review recoemmendation · 43c9137b
  Mostofa Patwary authored Mar 18, 2021
  
  43c9137b
18 Mar, 2021 1 commit
- refactored the fused kernels build · 0d5188c1
  Mohammad Shoeybi authored Mar 17, 2021
  
  0d5188c1
17 Mar, 2021 1 commit
- fixes to upper triangular masked softmax fusion kernel · 3b12ab15
  Vijay Korthikanti authored Mar 16, 2021
  
  3b12ab15
16 Mar, 2021 2 commits
- minor fixes · 531152d9
  Vijay Korthikanti authored Mar 16, 2021
  
  531152d9
- softmax data load/store optimization · b1a83375
  Vijay Korthikanti authored Mar 16, 2021
  
  b1a83375
10 Mar, 2021 1 commit
- ICT zeroshot evaluation code · 8b04e0e4
  Mostofa Patwary authored Mar 10, 2021
  
  8b04e0e4
08 Mar, 2021 1 commit
- Bfloat with fp32 grad acc · b4bc51b1
  Mohammad Shoeybi authored Mar 08, 2021
  
  b4bc51b1
04 Mar, 2021 3 commits
- Fix syntax · 57437cb1
  Rewon Child authored Mar 04, 2021
  
  57437cb1
- Update argument names and fix merge error · 0aff3629
  Rewon Child authored Mar 04, 2021
  
  0aff3629
- Update comments · 8676baca
  Rewon Child authored Mar 04, 2021
  
  8676baca
03 Mar, 2021 3 commits
- Remove pipeline stall timing (and avoid Explorer perf regressions) · ab09d819
  Deepak Narayanan authored Mar 03, 2021
  
  ab09d819
- Bring back call to ring_exchange() in _communicate() · cbad126a
  Deepak Narayanan authored Mar 03, 2021
  
  cbad126a
- Get PyTorch batched communication API working for interleaved schedule · 78cf869f
  Deepak Narayanan authored Feb 25, 2021
  
  78cf869f
26 Feb, 2021 2 commits
- Fix deadlock when get_num_microbatches() < pipeline-parallel size (don't try... · 08ddde0c
  Deepak Narayanan authored Feb 26, 2021
```
Fix deadlock when get_num_microbatches() < pipeline-parallel size (don't try to measure pipeline stall)
```
  08ddde0c
- Support Torch DDP for single-stage, num_microbatches() > 1 · 65eeb427
  Deepak Narayanan authored Feb 24, 2021
  
  65eeb427
25 Feb, 2021 1 commit
- Don't import deprecated model from realm_model which is broken. · c6f18ccf
  Jared Casper authored Feb 25, 2021
  
  c6f18ccf
23 Feb, 2021 3 commits
- Add option to log zeros · 4e77e7c6
  Rewon Child authored Feb 23, 2021
  
  4e77e7c6
- Added code for building embeddings and savings · bcd605f8
  Mostofa Patwary authored Feb 23, 2021
  
  bcd605f8
- fix warning condition · e055f3cf
  Vijay Korthikanti authored Feb 22, 2021
  
  e055f3cf
22 Feb, 2021 1 commit
- Fix interleaved schedule assertion · b7067cbd
  Deepak Narayanan authored Feb 22, 2021
  
  b7067cbd
18 Feb, 2021 2 commits
- addressed the comments · 3f3ba5e7
  Mostofa Patwary authored Feb 18, 2021
  
  3f3ba5e7
- fixed an issue with ICT · 3d96729a
  Mostofa Patwary authored Feb 17, 2021
  
  3d96729a