Commits · b46482e8d0c338e24177bb3c32bb4ea82ebff59a · OpenDAS / Megatron-LM

14 Sep, 2021 3 commits
- Fixes a bug in broadcasting that was causing hanging · b46482e8
  rprenger authored Sep 14, 2021
  
  b46482e8
- Addressing comments · 593b47b4
  rprenger authored Sep 14, 2021
  
  593b47b4
- Changing api to tokens_to_generate, making it so we always generate at least tokens_to_generate · 77979e3b
  rprenger authored Sep 14, 2021
  
  77979e3b
13 Sep, 2021 1 commit
- Changing the interface to the lm eval harness and fixing bugs caused by... · 42e83ee0
  rprenger authored Sep 13, 2021
```
Changing the interface to the lm eval harness and fixing bugs caused by misunderstanding out_seq_length
```
  42e83ee0
03 Sep, 2021 1 commit
- Reducing memory usage during inference · faf58b77
  rprenger authored Sep 03, 2021
  
  faf58b77
01 Sep, 2021 1 commit
- Fixing a bug caused by merge · d1b155c9
  rprenger authored Sep 01, 2021
  
  d1b155c9
31 Aug, 2021 1 commit
- Fixing merge conflict · 3d718bfc
  rprenger authored Aug 31, 2021
  
  3d718bfc
27 Aug, 2021 2 commits
- Adding API server · 3fe6821a
  Ryan Prenger authored Aug 27, 2021
  
  3fe6821a
- Added generate_samples_eval function · b6b7ba4d
  rprenger authored Aug 27, 2021
  
  b6b7ba4d
26 Aug, 2021 2 commits
- Found a bug. If you don't make this change and you ask for 1 token you get 2 etc. · 8694c7b0
  rprenger authored Aug 25, 2021
  
  8694c7b0
- Addressing comments · 055a673e
  rprenger authored Aug 25, 2021
  
  055a673e
12 Aug, 2021 1 commit
- Got it working on the full big model · 811183f0
  rprenger authored Aug 11, 2021
  
  811183f0
11 Aug, 2021 1 commit
- Got the probs piped · ddd36145
  rprenger authored Aug 11, 2021
  
  ddd36145
09 Aug, 2021 1 commit
- Fixing bug where the wrong token was used to index the log probabilities · 41df5ff7
  rprenger authored Aug 08, 2021
  
  41df5ff7
29 Jul, 2021 1 commit
- Outputting log probabilities · e0bf5199
  rprenger authored Jul 29, 2021
  
  e0bf5199
20 Jul, 2021 1 commit
- Made some simplifications that keep it from hanging · 59414b33
  rprenger authored Jul 20, 2021
  
  59414b33
19 Jul, 2021 1 commit
- Removing bug possibilities and adding timing info · 7a9c4a03
  rprenger authored Jul 19, 2021
  
  7a9c4a03
09 Jul, 2021 1 commit
- Refactoring code so server code is more independent of sampling and adding a... · 29dd0a35
  rprenger authored Jul 09, 2021
```
Refactoring code so server code is more independent of sampling and adding a CLI.  CLI still has URL of server hard-coded
```
  29dd0a35
02 Jul, 2021 1 commit
- Got 530 Billion parameter model working! · ac3db159
  rprenger authored Jul 02, 2021
  
  ac3db159
30 Jun, 2021 3 commits
- Removing unnecessary --recompute path · 453414da
  rprenger authored Jun 30, 2021
  
  453414da
- Clean up removing a lot of code. Works with curl people might still want a webpage or CLI · f7fe3865
  rprenger authored Jun 30, 2021
  
  f7fe3865
- Simpler broadcasting and some clean up · a9a3ef50
  rprenger authored Jun 30, 2021
  
  a9a3ef50
23 Jun, 2021 1 commit
- Working single sentence version · 2cdebf4b
  rprenger authored Jun 23, 2021
  
  2cdebf4b
21 Apr, 2021 3 commits
- addressed comments · e5ec27d7
  Mostofa Patwary authored Apr 21, 2021
  
  e5ec27d7
- addressed comments · 5a6431f5
  Mostofa Patwary authored Apr 21, 2021
  
  5a6431f5
- addressed comments · 5c2ce593
  Mostofa Patwary authored Apr 21, 2021
  
  5c2ce593
19 Apr, 2021 1 commit
- modified the params · 44bfcb32
  Mostofa Patwary authored Apr 19, 2021
  
  44bfcb32
16 Apr, 2021 1 commit
- added this function for evaluation · 045959cb
  Mostofa Patwary authored Apr 16, 2021
  
  045959cb
24 Mar, 2021 2 commits
- Fixing text generation and zeroshot eval and addressing comments. · 64a83fb5
  Jared Casper authored Mar 24, 2021
  
  64a83fb5
- Tasks seems to be working. · b938ec51
  Jared Casper authored Mar 23, 2021
  
  b938ec51
19 Dec, 2020 3 commits
- Fix text generation without recompute · 1d4e8760
  Jared Casper authored Dec 10, 2020
  
  1d4e8760
- Work batch-size name changes into task code · 3afcba6e
  Jared Casper authored Dec 09, 2020
  
  3afcba6e
- Initial implementation of pipelined text generation · 5c45db4a
  Jared Casper authored Dec 09, 2020
  
  5c45db4a
12 Nov, 2020 2 commits

Intra-layer MP -> Tensor MP, Inter-layer MP -> Pipeline MP · 52a5f2f2
Deepak Narayanan authored Oct 20, 2020

52a5f2f2

Pipeline parallelism implementation with periodic full-pipeline syncs · 7abd3e90

Deepak Narayanan authored Aug 29, 2020

Also includes following changes for inter-layer model-parallel implementation:
- Refactoring of model implementations
- Training loop changes to support inter-layer communication using `ring_exchange`
- New groups for inter-layer communication
- Checkpoint changes
- Command line arguments

7abd3e90

29 Sep, 2020 1 commit
- Text generation bugs · 1f3f834c
  Mostofa Patwary authored Sep 28, 2020
  
  1f3f834c
16 Apr, 2020 1 commit
- changed licence 2019 to 2020 · 463d1257
  Mohammad authored Apr 16, 2020
  
  463d1257
15 Apr, 2020 1 commit
- changed gpt2 masking to binary and masked_fill · 8ceed7c7
  Mohammad authored Apr 14, 2020
  
  8ceed7c7
14 Apr, 2020 1 commit
- Lint whole repo · b7f1b050
  Neel Kant authored Apr 14, 2020
  
  b7f1b050
03 Apr, 2020 1 commit
- sample generation runs · fffa0497
  Mohammad authored Apr 02, 2020
  
  fffa0497