Commits · f938e19a58f4e1a8a806902d138482650bf2a5ca · OpenDAS / Megatron-LM

20 Apr, 2021 3 commits
- added more comments · f938e19a
  Mostofa Patwary authored Apr 20, 2021
  
  f938e19a
- added more comments · c49b4644
  Mostofa Patwary authored Apr 20, 2021
  
  c49b4644
- Merge branch 'main' into main_dedup · 7c3d8b7a
  Mostofa Patwary authored Apr 20, 2021
  
  7c3d8b7a
12 Apr, 2021 2 commits
- Merge branch 'add_ref' into 'main' · f32a638d
  Jared Casper authored Apr 12, 2021
```
added link to the pipeline paper

See merge request ADLR/megatron-lm!259
```
  f32a638d
- added link to the pipeline papers · 9ec547c9
  mohammad authored Apr 11, 2021
  
  9ec547c9
08 Apr, 2021 2 commits
- Merge branch 'release_fixes' into 'main' · 8cfef1bf
  Deepak Narayanan authored Apr 08, 2021
```
Release fixes

See merge request ADLR/megatron-lm!258
```
  8cfef1bf
- Release fixes · 50a4b5fa
  Mohammad Shoeybi authored Apr 08, 2021
  
  50a4b5fa
03 Apr, 2021 3 commits
- Merge branch 'interleaved_bugfix' into 'main' · 23632ee5
  Mohammad Shoeybi authored Apr 03, 2021
```
Small bugfix to make sure refactored code works with interleaved schedule

See merge request ADLR/megatron-lm!256
```
  23632ee5
- Small bugfix to make sure refactored code works with interleaved schedule · 6fd78189
  Deepak Narayanan authored Apr 03, 2021
  
  6fd78189
- Merge branch 'pipeline_refactor' into 'main' · 3fc035d7
  Mohammad Shoeybi authored Apr 03, 2021
```
Pipeline refactor

See merge request ADLR/megatron-lm!254
```
  3fc035d7
02 Apr, 2021 3 commits
- Addressed MR comments, mostly adding comments to code. · e270f68a
  Jared Casper authored Apr 02, 2021
  
  e270f68a
- Merge branch 'main' into main_dedup · ee7b19e7
  Mostofa Patwary authored Apr 02, 2021
  
  ee7b19e7
- More features added · d413bd5f
  Mostofa Patwary authored Apr 01, 2021
  
  d413bd5f
01 Apr, 2021 1 commit
- updated filter_ngrams.py · f559787d
  Mostofa Patwary authored Mar 31, 2021
  
  f559787d
31 Mar, 2021 2 commits
- Merge branch 'bfloat_jit' into 'main' · f2d64c00
  Mohammad Shoeybi authored Mar 31, 2021
```
removed the checks for bfloat jitting

See merge request ADLR/megatron-lm!255
```
  f2d64c00
- removed the checks for bfloat jitting · d28716e8
  mshoeybi authored Mar 30, 2021
  
  d28716e8
30 Mar, 2021 1 commit
- fix one more issue · 0c01c2fe
  Mostofa Patwary authored Mar 29, 2021
  
  0c01c2fe
26 Mar, 2021 1 commit
- added parallelism for computing jaccard similaity · 43d307d5
  mpatwary authored Mar 25, 2021
  
  43d307d5
24 Mar, 2021 3 commits
- Fixing text generation and zeroshot eval and addressing comments. · 64a83fb5
  Jared Casper authored Mar 24, 2021
  
  64a83fb5
- Tasks seems to be working. · b938ec51
  Jared Casper authored Mar 23, 2021
  
  b938ec51
- pipeline code simplification · 3b91262e
  Vijay Korthikanti authored Mar 02, 2021
  
  3b91262e
23 Mar, 2021 1 commit

Merge branch 'extra_assertion' into 'main' · 2f3a2d68

Jared Casper authored Mar 23, 2021

Make sure pipeline-model-parallel size is greater than 2 for interleaved schedule

See merge request ADLR/megatron-lm!253

2f3a2d68

20 Mar, 2021 1 commit
- Make sure pipeline-model-parallel size is greater than 2 for interleaved schedule · 182841f7
  Deepak Narayanan authored Mar 20, 2021
  
  182841f7
19 Mar, 2021 5 commits
- Added more feature in train data deduplication · 882683dc
  Mostofa Patwary authored Mar 19, 2021
  
  882683dc
- Merge branch 'main_retriver_merge_ict_eval' into 'main' · a6e00d97
  Jared Casper authored Mar 19, 2021
```
ICT zeroshot evaluation

See merge request ADLR/megatron-lm!248
```
  a6e00d97
- ICT zeroshot evaluation · fcfd0949
  Mostofa Patwary authored Mar 19, 2021
  
  fcfd0949
- Merge branch 'bfloat_fused_softmax' into 'main' · c5346794
  Jared Casper authored Mar 19, 2021
```
Bfloat fused softmax + fused layer norm

See merge request ADLR/megatron-lm!251
```
  c5346794
- Bfloat fused softmax + fused layer norm · 0fa7175f
  Mohammad Shoeybi authored Mar 19, 2021
  
  0fa7175f
18 Mar, 2021 2 commits
- Merge branch 'ninja_compilation_fix' into 'main' · d9b1c681
  Jared Casper authored Mar 18, 2021
```
refactored the fused kernels build

See merge request ADLR/megatron-lm!250
```
  d9b1c681
- refactored the fused kernels build · 0d5188c1
  Mohammad Shoeybi authored Mar 17, 2021
  
  0d5188c1
17 Mar, 2021 2 commits
- Merge branch 'softmax_perf' into 'main' · 876096d5
  Jared Casper authored Mar 17, 2021
```
softmax data load/store optimization

See merge request ADLR/megatron-lm!249
```
  876096d5
- fixes to upper triangular masked softmax fusion kernel · 3b12ab15
  Vijay Korthikanti authored Mar 16, 2021
  
  3b12ab15
16 Mar, 2021 2 commits
- minor fixes · 531152d9
  Vijay Korthikanti authored Mar 16, 2021
  
  531152d9
- softmax data load/store optimization · b1a83375
  Vijay Korthikanti authored Mar 16, 2021
  
  b1a83375
08 Mar, 2021 3 commits
- Merge branch 'bfloat_with_fp32_grad_acc' into 'main' · 1a2cb60c
  Jared Casper authored Mar 08, 2021
```
Bfloat with fp32 grad acc

See merge request ADLR/megatron-lm!247
```
  1a2cb60c
- Bfloat with fp32 grad acc · b4bc51b1
  Mohammad Shoeybi authored Mar 08, 2021
  
  b4bc51b1
- Added code to cleanup and fix code · 47e7eb80
  Mostofa Patwary authored Mar 08, 2021
  
  47e7eb80
05 Mar, 2021 3 commits
- Fixed a bug · 9c2eb971
  Mostofa Patwary authored Mar 05, 2021
  
  9c2eb971
- Many more features added · b35e15f2
  Mostofa Patwary authored Mar 04, 2021
  
  b35e15f2
- Many more features added · c44f7622
  Mostofa Patwary authored Mar 04, 2021
  
  c44f7622