Commits · 6495927e7ccf4a67720fd81fe5013f752a71f45c · OpenDAS / Megatron-LM

22 Jun, 2020 1 commit
- Address most of comments from code review w/ Mohammad · 6495927e
  Neel Kant authored Jun 22, 2020
  
  6495927e
05 Jun, 2020 1 commit
- Prune changes to only be related to ICT · 32bb4edc
  Neel Kant authored Jun 05, 2020
  
  32bb4edc
27 May, 2020 1 commit
- Correct CrossEntropyLoss · 0e8f4331
  Neel Kant authored May 27, 2020
  
  0e8f4331
07 May, 2020 1 commit
- Add REALMAnswerSpanModel and MLM features · f2094783
  Neel Kant authored May 06, 2020
  
  f2094783
03 May, 2020 1 commit
- Move get_train_val... to dataset_utils · 16a64c41
  Neel Kant authored May 03, 2020
  
  16a64c41
30 Apr, 2020 1 commit
- Whitening code · 9d225b44
  Neel Kant authored Apr 29, 2020
  
  9d225b44
29 Apr, 2020 1 commit
- Update and test misc functionality · 6c0a5bd8
  Neel Kant authored Apr 28, 2020
  
  6c0a5bd8
24 Apr, 2020 1 commit
- Write pretrain_realm.py and misc dataset_type left from earlier · f7f730e1
  Neel Kant authored Apr 23, 2020
  
  f7f730e1
21 Apr, 2020 1 commit
- Simplify batch and forward for ICT dataset and model · 5235ed87
  Neel Kant authored Apr 21, 2020
  
  5235ed87
17 Apr, 2020 1 commit
- Refactor before merge request · 4abd7ce2
  Neel Kant authored Apr 17, 2020
  
  4abd7ce2
15 Apr, 2020 2 commits
- Debug hash dump · 662dc982
  Neel Kant authored Apr 15, 2020
  
  662dc982
- Implement reformer hashing scheme · 81c71789
  Neel Kant authored Apr 15, 2020
  
  81c71789
14 Apr, 2020 1 commit
- Re-adjust dataloader properties · 064a6881
  Neel Kant authored Apr 13, 2020
  
  064a6881
07 Apr, 2020 1 commit
- Fix issue with validation dataloader · 064bdc46
  Neel Kant authored Apr 07, 2020
  
  064bdc46
03 Apr, 2020 2 commits
- Implement InverseClozeDataset with IndexedDataset · 4558e42f
  Neel Kant authored Apr 03, 2020
  
  4558e42f
- Fix InverseClozeDataset behavior (with commented out test code) · ca6b6687
  Neel Kant authored Apr 03, 2020
  
  ca6b6687
31 Mar, 2020 1 commit
- Remove debug statements and correct dataloader · 6f56b909
  Neel Kant authored Mar 31, 2020
  
  6f56b909
30 Mar, 2020 1 commit
- Add debug statements · f1b2524b
  Neel Kant authored Mar 30, 2020
  
  f1b2524b
27 Mar, 2020 1 commit
- pretrain_bert_icy.py compiles and runs · 1dd51c0e
  Neel Kant authored Mar 27, 2020
  
  1dd51c0e
26 Mar, 2020 1 commit
- Modify pretrain_bert_ict.py to work with ICTBertModel · b1efc33d
  Neel Kant authored Mar 26, 2020
  
  b1efc33d
24 Mar, 2020 1 commit
- Correct some args and create pretrain_bert_ict.py · 21a916b1
  Neel Kant authored Mar 24, 2020
  
  21a916b1
17 Mar, 2020 2 commits
- moved albert to bert · 09e05c6f
  Mohammad Shoeybi authored Mar 16, 2020
  
  09e05c6f
- moved pretrain albert to pretrain bert · 3e4e1ab2
  Mohammad Shoeybi authored Mar 16, 2020
  
  3e4e1ab2
14 Jan, 2020 1 commit
- added query-key layer scaling and softmax fp32 option · 691747b1
  Mohammad Shoeybi authored Jan 13, 2020
  
  691747b1
21 Nov, 2019 1 commit
- fixed padding issue · f6a6811f
  Mohammad Shoeybi authored Nov 20, 2019
  
  f6a6811f
19 Nov, 2019 1 commit
- Plumbing for new albert dataset, including mods to arguments for data loaders. · 1a1cfcff
  Jared Casper authored Nov 18, 2019
  
  1a1cfcff
31 Oct, 2019 1 commit
- refacotred for code reuse · 34be7dd3
  Mohammad Shoeybi authored Oct 30, 2019
  
  34be7dd3
29 Oct, 2019 1 commit
- Major refactoring, combining gpt2 and bert · 73af1290
  Mohammad Shoeybi authored Oct 29, 2019
  
  73af1290
09 Oct, 2019 1 commit
- moved padding to utils · 2e6d5ed9
  Mohammad Shoeybi authored Oct 09, 2019
  
  2e6d5ed9
08 Oct, 2019 3 commits
- moved few common elements between bert and gpt to utils · 787c1a0b
  Mohammad Shoeybi authored Oct 08, 2019
  
  787c1a0b
- created megatron package · b886b7bb
  Mohammad Shoeybi authored Oct 08, 2019
  
  b886b7bb
- Use DDP command line argument instead of source flag in pretrain_bert.py. · 72c5f666
  Jared Casper authored Oct 07, 2019
```
Note that there is currently an issue with bert using Torch DDP.
```
  72c5f666
04 Oct, 2019 1 commit
- Updating public repo with latest changes. · a1d04b79
  Jared Casper authored Oct 04, 2019
  
  a1d04b79
13 Aug, 2019 1 commit
- eval+numeric update · a0368ddf
  Raul Puri authored Aug 12, 2019
  
  a0368ddf
30 Jul, 2019 1 commit

large update including model parallelism and gpt2 · abe36e2e

Raul Puri authored Jul 29, 2019


Co-authored-by: shoeybi <shoeybim@gmail.com>
Co-authored-by: raulpuric <raulpuric@berkeley.edu>
Co-authored-by: jaredcasper <jaredcasper@gmail.com>
Co-authored-by: mpatwary <mostofa.patwary@gmail.com>
Co-authored-by: plegresl <plegresl@gmail.com>

abe36e2e

23 Apr, 2019 1 commit

Faster dataloader merge () · 66719e97

Raul Puri authored Apr 23, 2019

* threaded tf_dl+presplit sentences+shuffled dataset with resume

* elaborate in readme

66719e97

27 Mar, 2019 1 commit
- initial commit · fb4cbdc2
  Raul Puri authored Mar 27, 2019
  
  fb4cbdc2