Commits · 89e8d27e4f749a6fe620798d45709bb050ed7abc · OpenDAS / Megatron-LM

27 Aug, 2021 3 commits
- Merge branch 'server' into 'main' · 89e8d27e
  Jared Casper authored Aug 27, 2021
```
Adding API server

See merge request ADLR/megatron-lm!294
```
  89e8d27e
- Adding API server · 3fe6821a
  Ryan Prenger authored Aug 27, 2021
  
  3fe6821a
- Merge branch 'dist_chkpt_act' into 'main' · 136d63cb
  Jared Casper authored Aug 27, 2021
```
Revisited distributing checkpointed activations along the tensor parallel ranks

See merge request ADLR/megatron-lm!311
```
  136d63cb
24 Aug, 2021 1 commit
- Merge branch 'github_fused_softmax' into 'main' · 0be40526
  Jared Casper authored Aug 23, 2021
```
Fused softmax checks and additions from Github (#133)

See merge request ADLR/megatron-lm!312
```
  0be40526
23 Aug, 2021 7 commits
- removed unused variable · bc7b3539
  mshoeybi authored Aug 22, 2021
  
  bc7b3539
- Modify comments · b0b017ff
  hyunwoongko authored Aug 18, 2021
  
  b0b017ff
- chagne PR by reviews · c6b77639
  hyunwoongko authored Aug 18, 2021
  
  c6b77639
- fix bugs in fused softmax · 0d350c8d
  hyunwoongko authored Aug 13, 2021
  
  0d350c8d
- Remove useless include · 116820a5
  hyunwoongko authored Aug 13, 2021
  
  116820a5
- Improve and fix bugs about fused softmax layer · 40becfc9
  hyunwoongko authored Aug 13, 2021
  
  40becfc9
- tested · cb5e611d
  mshoeybi authored Aug 22, 2021
  
  cb5e611d
21 Aug, 2021 5 commits
- some cleanup · c61dc22f
  mshoeybi authored Aug 21, 2021
  
  c61dc22f
- added for pp · b8940b96
  mshoeybi authored Aug 21, 2021
  
  b8940b96
- resolved conflicts · 7f2cc3a4
  mshoeybi authored Aug 20, 2021
  
  7f2cc3a4
- resolved conflicts · 30b92cf5
  mshoeybi authored Aug 20, 2021
  
  30b92cf5
- resolved conflicts · 8cb389b8
  mshoeybi authored Aug 20, 2021
  
  8cb389b8
19 Aug, 2021 9 commits
- Merge branch 'slym/partial_chpt' into 'main' · 23266c57
  Jared Casper authored Aug 19, 2021
```
Checkpoint a set number of individual Transformer layers

See merge request ADLR/megatron-lm!301
```
  23266c57
- Merge branch 'mem_opt_for_contig_buffer' into 'main' · 3715b0bc
  Jared Casper authored Aug 19, 2021
```
Memory optimizations for contiguous buffers

See merge request ADLR/megatron-lm!310
```
  3715b0bc
- update readme and arguement definition · 99f47676
  slym authored Aug 19, 2021
  
  99f47676
- more updates on examples · 6985e589
  slym authored Aug 18, 2021
  
  6985e589
- Checkpoint a set number of invidividual Transformer layers · c1e0689d
  slym authored Aug 10, 2021
```
consider the case of pipeline-model prallelism

clean up arugments

argument naming cleanup

update readme and examples
```
  c1e0689d
- onlly support pp=1 · 7b585440
  mshoeybi authored Aug 19, 2021
  
  7b585440
- removed contiguous buffer for checkpointed activation · e923ec52
  mshoeybi authored Aug 19, 2021
  
  e923ec52
- pushed a fix for torch ddp · 6a0ef5b1
  mshoeybi authored Aug 18, 2021
  
  6a0ef5b1
- made contiguous buffer in local ddp default · e8fb052f
  mshoeybi authored Aug 18, 2021
  
  e8fb052f
18 Aug, 2021 3 commits
- set grads to none for the contig. buffers · df6e3cd7
  mshoeybi authored Aug 18, 2021
  
  df6e3cd7
- Merge branch 'lmcafee/empty-cache' into 'main' · 68797d90
  Jared Casper authored Aug 18, 2021
```
bug fix for previous MR (Flag to call empty_cache() each iteration, to reduce fragmentation)

See merge request ADLR/megatron-lm!309
```
  68797d90
- fixed flag name bug · db08a636
  Lawrence McAfee authored Aug 18, 2021
  
  db08a636
17 Aug, 2021 12 commits
- Merge branch 'github_small_prs' into 'main' · 8e8d7603
  Jared Casper authored Aug 17, 2021
```
minor changes from github issues

See merge request ADLR/megatron-lm!308
```
  8e8d7603
- minor changes from github issues · 60078cf3
  mshoeybi authored Aug 17, 2021
  
  60078cf3
- Merge branch 'lmcafee/empty-cache' into 'main' · bd315c35
  Jared Casper authored Aug 17, 2021
```
Flag to call empty_cache() each iteration, to reduce fragmentation

See merge request ADLR/megatron-lm!306
```
  bd315c35
- Merge branch 'iter_read_update' into 'main' · 5ca20cdd
  Jared Casper authored Aug 17, 2021
```
simplified the iteration read check across ranks

See merge request ADLR/megatron-lm!307
```
  5ca20cdd
- updated argument name · 52b2296b
  Lawrence McAfee authored Aug 17, 2021
  
  52b2296b
- simplified the iteration read check across ranks · ede0a58f
  mshoeybi authored Aug 17, 2021
  
  ede0a58f
- Merge branch 'github_small_prs' into 'main' · a8f4edcb
  Jared Casper authored Aug 17, 2021
```
some small PRs from github

See merge request ADLR/megatron-lm!305
```
  a8f4edcb
- Merge branch 'leftover_fixes' into 'main' · 6ee3d210
  Jared Casper authored Aug 17, 2021
```
added across ranks sync for checkpoint iteration load and couple of other fixes

See merge request ADLR/megatron-lm!304
```
  6ee3d210
- Merge branch 'typo' of https://github.com/hwijeen/Megatron-LM into github_small_prs · f69f699f
  mshoeybi authored Aug 16, 2021
  
  f69f699f
- Merge branch 'update-sample-bert-pretrain-args' of... · 2ef470e8
  mshoeybi authored Aug 16, 2021
```
Merge branch 'update-sample-bert-pretrain-args' of https://github.com/roclark/Megatron-LM into github_small_prs
```
  2ef470e8
- Merge branch 'patch-1' of https://github.com/eric-haibin-lin/Megatron-LM into github_small_prs · 2c26bf78
  mshoeybi authored Aug 16, 2021
  
  2c26bf78
- Merge branch 'main' of https://github.com/ktaebum/Megatron-LM into github_small_prs · 562d1b21
  mshoeybi authored Aug 16, 2021
  
  562d1b21