Commits · b46482e8d0c338e24177bb3c32bb4ea82ebff59a · OpenDAS / Megatron-LM

14 Sep, 2021 3 commits
- Fixes a bug in broadcasting that was causing hanging · b46482e8
  rprenger authored Sep 14, 2021
  
  b46482e8
- Addressing comments · 593b47b4
  rprenger authored Sep 14, 2021
  
  593b47b4
- Changing api to tokens_to_generate, making it so we always generate at least tokens_to_generate · 77979e3b
  rprenger authored Sep 14, 2021
  
  77979e3b
13 Sep, 2021 1 commit
- Changing the interface to the lm eval harness and fixing bugs caused by... · 42e83ee0
  rprenger authored Sep 13, 2021
```
Changing the interface to the lm eval harness and fixing bugs caused by misunderstanding out_seq_length
```
  42e83ee0
07 Sep, 2021 1 commit
- remove increasing nccl stream for overlapping allreduce and gemm · f3e57f6f
  Sangkug Lym authored Sep 06, 2021
  
  f3e57f6f
03 Sep, 2021 2 commits
- Reducing memory usage during inference · faf58b77
  rprenger authored Sep 03, 2021
  
  faf58b77
- reflect feedback · 4df8b7a2
  slym authored Sep 02, 2021
  
  4df8b7a2
02 Sep, 2021 3 commits
- reflect feedback · 3f652469
  slym authored Sep 02, 2021
  
  3f652469
- minor changes · 16c90445
  slym authored Sep 02, 2021
  
  16c90445
- t # This is a combination of 2 commits. · cf7efd4f
  Sangkug Lym authored Aug 30, 2021
```
allreduce overlap with wgrad gemm

change custom delay to dummy add
```
  cf7efd4f
01 Sep, 2021 1 commit
- Fixing a bug caused by merge · d1b155c9
  rprenger authored Sep 01, 2021
  
  d1b155c9
31 Aug, 2021 1 commit
- Fixing merge conflict · 3d718bfc
  rprenger authored Aug 31, 2021
  
  3d718bfc
27 Aug, 2021 3 commits
- Use nvfuser at pytorch >= 1.10 · dba2506d
  Sangkug Lym authored Aug 27, 2021
  
  dba2506d
- Adding API server · 3fe6821a
  Ryan Prenger authored Aug 27, 2021
  
  3fe6821a
- Added generate_samples_eval function · b6b7ba4d
  rprenger authored Aug 27, 2021
  
  b6b7ba4d
26 Aug, 2021 3 commits
- Fixing bug caused by merge · 448cb299
  rprenger authored Aug 25, 2021
  
  448cb299
- Found a bug. If you don't make this change and you ask for 1 token you get 2 etc. · 8694c7b0
  rprenger authored Aug 25, 2021
  
  8694c7b0
- Addressing comments · 055a673e
  rprenger authored Aug 25, 2021
  
  055a673e
23 Aug, 2021 7 commits
- removed unused variable · bc7b3539
  mshoeybi authored Aug 22, 2021
  
  bc7b3539
- Modify comments · b0b017ff
  hyunwoongko authored Aug 18, 2021
  
  b0b017ff
- chagne PR by reviews · c6b77639
  hyunwoongko authored Aug 18, 2021
  
  c6b77639
- fix bugs in fused softmax · 0d350c8d
  hyunwoongko authored Aug 13, 2021
  
  0d350c8d
- Remove useless include · 116820a5
  hyunwoongko authored Aug 13, 2021
  
  116820a5
- Improve and fix bugs about fused softmax layer · 40becfc9
  hyunwoongko authored Aug 13, 2021
  
  40becfc9
- tested · cb5e611d
  mshoeybi authored Aug 22, 2021
  
  cb5e611d
21 Aug, 2021 4 commits
- some cleanup · c61dc22f
  mshoeybi authored Aug 21, 2021
  
  c61dc22f
- added for pp · b8940b96
  mshoeybi authored Aug 21, 2021
  
  b8940b96
- resolved conflicts · 7f2cc3a4
  mshoeybi authored Aug 20, 2021
  
  7f2cc3a4
- resolved conflicts · 30b92cf5
  mshoeybi authored Aug 20, 2021
  
  30b92cf5
19 Aug, 2021 6 commits
- update readme and arguement definition · 99f47676
  slym authored Aug 19, 2021
  
  99f47676
- Checkpoint a set number of invidividual Transformer layers · c1e0689d
  slym authored Aug 10, 2021
```
consider the case of pipeline-model prallelism

clean up arugments

argument naming cleanup

update readme and examples
```
  c1e0689d
- onlly support pp=1 · 7b585440
  mshoeybi authored Aug 19, 2021
  
  7b585440
- removed contiguous buffer for checkpointed activation · e923ec52
  mshoeybi authored Aug 19, 2021
  
  e923ec52
- pushed a fix for torch ddp · 6a0ef5b1
  mshoeybi authored Aug 18, 2021
  
  6a0ef5b1
- made contiguous buffer in local ddp default · e8fb052f
  mshoeybi authored Aug 18, 2021
  
  e8fb052f
18 Aug, 2021 2 commits
- set grads to none for the contig. buffers · df6e3cd7
  mshoeybi authored Aug 18, 2021
  
  df6e3cd7
- fixed flag name bug · db08a636
  Lawrence McAfee authored Aug 18, 2021
  
  db08a636
17 Aug, 2021 3 commits
- minor changes from github issues · 60078cf3
  mshoeybi authored Aug 17, 2021
  
  60078cf3
- updated argument name · 52b2296b
  Lawrence McAfee authored Aug 17, 2021
  
  52b2296b
- simplified the iteration read check across ranks · ede0a58f
  mshoeybi authored Aug 17, 2021
  
  ede0a58f