Commits · aaa5715a8167862bee0bdcc248d64e905fd40b74 · OpenDAS / Megatron-LM

16 Aug, 2022 3 commits
- fixed grad scalar warning for bf16 · aaa5715a
  Mohammad Shoeybi authored Aug 16, 2022
  
  aaa5715a
- Merge branch 'fix_grad_scalar_warning' into 'main' · 91384a5a
  Jared Casper authored Aug 15, 2022
```
fixed grad scalar warning so it only prints it for fp16

See merge request ADLR/megatron-lm!441
```
  91384a5a
- fixed grad scalar warning so it only prints it for fp16 · 27bc1338
  mshoeybi authored Aug 15, 2022
  
  27bc1338
10 Aug, 2022 2 commits
- Merge branch 'timing' into 'main' · d2073912
  Jared Casper authored Aug 10, 2022
```
Timing levels

See merge request ADLR/megatron-lm!436
```
  d2073912
- Timing levels · 77efcccb
  Mohammad Shoeybi authored Aug 10, 2022
  
  77efcccb
06 Aug, 2022 2 commits
- Merge branch 'beam_search' into 'main' · 83d78670
  Jared Casper authored Aug 05, 2022
```
fix a bug for size mismatch

See merge request ADLR/megatron-lm!438
```
  83d78670
- fix a bug for size mismatch · b7b2d6a9
  Peng Xu authored Aug 05, 2022
  
  b7b2d6a9
29 Jul, 2022 1 commit

Merge branch 'fused_softmax_kernel_fixes' into 'main' · 189e72a7

Jared Casper authored Jul 29, 2022

support for all mask in fused kernel + avoiding inplace operation in bwd pass

See merge request ADLR/megatron-lm!435

189e72a7

28 Jul, 2022 1 commit
- support for all mask in fused kernel + avoiding inplace operation in bwd pass · 76db9583
  Vijay Korthikanti authored Jul 28, 2022
  
  76db9583
27 Jul, 2022 1 commit

Merge branch 'add_ring_exchange_flag' into 'main' · 8df49e72

Jared Casper authored Jul 26, 2022

added a flag to be able to switch between pytorch and ring exchange p2p

See merge request ADLR/megatron-lm!434

8df49e72

26 Jul, 2022 3 commits
- added a flag to be able to switch between pytorch and ring exchange p2p · e36cdd79
  Mohammad Shoeybi authored Jul 26, 2022
  
  e36cdd79
- Merge branch 'del_merge' into 'main' · c464a103
  Jared Casper authored Jul 26, 2022
```
Remove old merge tool.

See merge request ADLR/megatron-lm!433
```
  c464a103
- Remove old merge tool. · 5df9e1fb
  Jared Casper authored Jul 26, 2022
  
  5df9e1fb
20 Jul, 2022 2 commits

Merge branch 'oom' into 'main' · 0bb597b4

Jared Casper authored Jul 20, 2022

Adding checks for total number of tokens to keep server from crashing

See merge request ADLR/megatron-lm!428

0bb597b4

Merge branch 'factual' into 'main' · 2bc3c1a4

Jared Casper authored Jul 20, 2022

Implements the top_p decay and top_p bound parameters so from the Factual Sampling work

See merge request ADLR/megatron-lm!423

2bc3c1a4

19 Jul, 2022 5 commits
- Merge branch 'lmcafee/distop-readme' into 'main' · badcb5a7
  Jared Casper authored Jul 19, 2022
```
Distributed optimizer readme section.

See merge request ADLR/megatron-lm!429
```
  badcb5a7
- added title to ZeRO paper. · ff8ce3db
  Jared Casper authored Jul 19, 2022
  
  ff8ce3db
- fixes. · 885a0428
  Lawrence McAfee authored Jul 19, 2022
  
  885a0428
- added dist-op section. · 69e25145
  Lawrence McAfee authored Jul 19, 2022
  
  69e25145
- added activation checkpointing link. · c80b4230
  Lawrence McAfee authored Jul 19, 2022
  
  c80b4230
18 Jul, 2022 4 commits
- Adding checks for total number of tokens to keep server from crashing · cfe2c2be
  rprenger authored Jul 18, 2022
  
  cfe2c2be
- Merging with main and fixing merge conflict · 2eea6216
  rprenger authored Jul 18, 2022
  
  2eea6216
- Merge branch 'checkpoint_util' into 'main' · 5f694372
  Jared Casper authored Jul 18, 2022
```
Checkpoint conversion utility

See merge request ADLR/megatron-lm!289
```
  5f694372
- Merge branch 'beam_search' into 'main' · bdbb30fb
  Jared Casper authored Jul 18, 2022
```
Fix bugs for beam search when using pipeline parallelization

See merge request ADLR/megatron-lm!426
```
  bdbb30fb
15 Jul, 2022 3 commits
- remove debug lines for printing · 2e65bee5
  peng xu authored Jul 15, 2022
  
  2e65bee5
- Merge branch 'main' into beam_search · b8428a7f
  peng xu authored Jul 15, 2022
  
  b8428a7f
- fix pipeline parallel for beam search · e5034150
  peng xu authored Jul 15, 2022
  
  e5034150
12 Jul, 2022 1 commit
- Add error checking by wrapping all weights and biases into named messages with named tensors. · 748a9a7d
  Jared Casper authored Jul 11, 2022
  
  748a9a7d
24 Jun, 2022 6 commits
- Merge branch 'main' into checkpoint_util · 2755bcb8
  Jared Casper authored Jun 24, 2022
  
  2755bcb8
- Merge branch 'ci' into 'main' · 3f4e71df
  Jared Casper authored Jun 24, 2022
```
Update ci image.

See merge request ADLR/megatron-lm!425
```
  3f4e71df
- Update ci image. · e075d830
  Jared Casper authored Jun 24, 2022
  
  e075d830
- Fixing up checkpointing. · 2725dc0b
  Jared Casper authored Jun 24, 2022
  
  2725dc0b
- Remove unused function. · 4eb802c4
  Jared Casper authored Jun 24, 2022
  
  4eb802c4
- Merge branch 'main' into checkpoint_util · 80389ef6
  Jared Casper authored Jun 24, 2022
  
  80389ef6
21 Jun, 2022 1 commit
- Changing name of parameters from factual_decay to top_p_decay · ed6806ac
  rprenger authored Jun 21, 2022
  
  ed6806ac
08 Jun, 2022 5 commits
- Adding top_p decay and bound for factual sampling from Factuality Enhanced LMs · b49349ec
  rprenger authored Jun 08, 2022
  
  b49349ec
- Merge branch 'lmcafee/distrib-opt-nodupe' into 'main' · d07d29df
  Jared Casper authored Jun 08, 2022
```
Distributed optimizer

See merge request ADLR/megatron-lm!408
```
  d07d29df
- fixed load_checkpoint(). · 74ba3797
  Lawrence McAfee authored Jun 08, 2022
  
  74ba3797
- error fixes & tested. · b178e6fc
  Lawrence McAfee authored Jun 08, 2022
  
  b178e6fc
- added backwards compatibility to checkpointing.py. · 977efdfb
  Lawrence McAfee authored Jun 08, 2022
  
  977efdfb