Commits · 1446bb64322835ed0dc94d66b5bc2f1d769afd75 · OpenDAS / Megatron-LM

17 Mar, 2020 2 commits
- moved albert to bert · 09e05c6f
  Mohammad Shoeybi authored Mar 16, 2020
  
  09e05c6f
- moved pretrain albert to pretrain bert · 3e4e1ab2
  Mohammad Shoeybi authored Mar 16, 2020
  
  3e4e1ab2
14 Jan, 2020 1 commit
- added query-key layer scaling and softmax fp32 option · 691747b1
  Mohammad Shoeybi authored Jan 13, 2020
  
  691747b1
21 Nov, 2019 1 commit
- fixed padding issue · f6a6811f
  Mohammad Shoeybi authored Nov 20, 2019
  
  f6a6811f
19 Nov, 2019 1 commit
- Plumbing for new albert dataset, including mods to arguments for data loaders. · 1a1cfcff
  Jared Casper authored Nov 18, 2019
  
  1a1cfcff
31 Oct, 2019 1 commit
- refacotred for code reuse · 34be7dd3
  Mohammad Shoeybi authored Oct 30, 2019
  
  34be7dd3
29 Oct, 2019 1 commit
- Major refactoring, combining gpt2 and bert · 73af1290
  Mohammad Shoeybi authored Oct 29, 2019
  
  73af1290
09 Oct, 2019 1 commit
- moved padding to utils · 2e6d5ed9
  Mohammad Shoeybi authored Oct 09, 2019
  
  2e6d5ed9
08 Oct, 2019 3 commits
- moved few common elements between bert and gpt to utils · 787c1a0b
  Mohammad Shoeybi authored Oct 08, 2019
  
  787c1a0b
- created megatron package · b886b7bb
  Mohammad Shoeybi authored Oct 08, 2019
  
  b886b7bb
- Use DDP command line argument instead of source flag in pretrain_bert.py. · 72c5f666
  Jared Casper authored Oct 07, 2019
```
Note that there is currently an issue with bert using Torch DDP.
```
  72c5f666
04 Oct, 2019 1 commit
- Updating public repo with latest changes. · a1d04b79
  Jared Casper authored Oct 04, 2019
  
  a1d04b79
13 Aug, 2019 1 commit
- eval+numeric update · a0368ddf
  Raul Puri authored Aug 12, 2019
  
  a0368ddf
30 Jul, 2019 1 commit

large update including model parallelism and gpt2 · abe36e2e

Raul Puri authored Jul 29, 2019


Co-authored-by: shoeybi <shoeybim@gmail.com>
Co-authored-by: raulpuric <raulpuric@berkeley.edu>
Co-authored-by: jaredcasper <jaredcasper@gmail.com>
Co-authored-by: mpatwary <mostofa.patwary@gmail.com>
Co-authored-by: plegresl <plegresl@gmail.com>

abe36e2e

23 Apr, 2019 1 commit

Faster dataloader merge () · 66719e97

Raul Puri authored Apr 23, 2019

* threaded tf_dl+presplit sentences+shuffled dataset with resume

* elaborate in readme

66719e97

27 Mar, 2019 1 commit
- initial commit · fb4cbdc2
  Raul Puri authored Mar 27, 2019
  
  fb4cbdc2