Commits · dedb2ef78cdf54a54b6986ac0e8fc0045dd4fa07 · OpenDAS / Megatron-LM

30 Mar, 2020 2 commits
- removed building tokenizer from bert dataset · dedb2ef7
  Mohammad authored Mar 30, 2020
  
  dedb2ef7
- both bert and gpt2 tested and working · 1788c910
  Mohammad authored Mar 29, 2020
  
  1788c910
29 Mar, 2020 3 commits
- refactored pretrain-bert · 3366a5b0
  Mohammad authored Mar 28, 2020
  
  3366a5b0
- refactored training · 27e14f82
  Mohammad authored Mar 28, 2020
  
  27e14f82
- utils partially refactored and tested · 3f58649b
  Mohammad authored Mar 28, 2020
  
  3f58649b
28 Mar, 2020 2 commits
- working on utils · 5050203f
  Mohammad authored Mar 28, 2020
  
  5050203f
- arguments.py refactored · 94e2ca57
  Mohammad authored Mar 27, 2020
  
  94e2ca57
27 Mar, 2020 1 commit
- working on bert · 599e959a
  Mohammad authored Mar 26, 2020
  
  599e959a
17 Mar, 2020 2 commits
- moved albert to bert · 09e05c6f
  Mohammad Shoeybi authored Mar 16, 2020
  
  09e05c6f
- moved pretrain albert to pretrain bert · 3e4e1ab2
  Mohammad Shoeybi authored Mar 16, 2020
  
  3e4e1ab2
14 Jan, 2020 1 commit
- added query-key layer scaling and softmax fp32 option · 691747b1
  Mohammad Shoeybi authored Jan 13, 2020
  
  691747b1
21 Nov, 2019 1 commit
- fixed padding issue · f6a6811f
  Mohammad Shoeybi authored Nov 20, 2019
  
  f6a6811f
19 Nov, 2019 1 commit
- Plumbing for new albert dataset, including mods to arguments for data loaders. · 1a1cfcff
  Jared Casper authored Nov 18, 2019
  
  1a1cfcff
31 Oct, 2019 1 commit
- refacotred for code reuse · 34be7dd3
  Mohammad Shoeybi authored Oct 30, 2019
  
  34be7dd3
29 Oct, 2019 1 commit
- Major refactoring, combining gpt2 and bert · 73af1290
  Mohammad Shoeybi authored Oct 29, 2019
  
  73af1290
09 Oct, 2019 1 commit
- moved padding to utils · 2e6d5ed9
  Mohammad Shoeybi authored Oct 09, 2019
  
  2e6d5ed9
08 Oct, 2019 3 commits
- moved few common elements between bert and gpt to utils · 787c1a0b
  Mohammad Shoeybi authored Oct 08, 2019
  
  787c1a0b
- created megatron package · b886b7bb
  Mohammad Shoeybi authored Oct 08, 2019
  
  b886b7bb
- Use DDP command line argument instead of source flag in pretrain_bert.py. · 72c5f666
  Jared Casper authored Oct 07, 2019
```
Note that there is currently an issue with bert using Torch DDP.
```
  72c5f666
04 Oct, 2019 1 commit
- Updating public repo with latest changes. · a1d04b79
  Jared Casper authored Oct 04, 2019
  
  a1d04b79
13 Aug, 2019 1 commit
- eval+numeric update · a0368ddf
  Raul Puri authored Aug 12, 2019
  
  a0368ddf
30 Jul, 2019 1 commit

large update including model parallelism and gpt2 · abe36e2e

Raul Puri authored Jul 29, 2019


Co-authored-by: shoeybi <shoeybim@gmail.com>
Co-authored-by: raulpuric <raulpuric@berkeley.edu>
Co-authored-by: jaredcasper <jaredcasper@gmail.com>
Co-authored-by: mpatwary <mostofa.patwary@gmail.com>
Co-authored-by: plegresl <plegresl@gmail.com>

abe36e2e

23 Apr, 2019 1 commit

Faster dataloader merge () · 66719e97

Raul Puri authored Apr 23, 2019

* threaded tf_dl+presplit sentences+shuffled dataset with resume

* elaborate in readme

66719e97

27 Mar, 2019 1 commit
- initial commit · fb4cbdc2
  Raul Puri authored Mar 27, 2019
  
  fb4cbdc2