Commits · 22e3c7e60726331ee1451fddf831012a1ae65a7b · OpenDAS / Megatron-LM

05 Jun, 2020 3 commits
- added fp16 cross entropy loss option for gpt2 · 22e3c7e6
  mohammad authored Jun 05, 2020
  
  22e3c7e6
- added fp16 cross entropy loss option for gpt2 · acfe848e
  mohammad authored Jun 05, 2020
  
  acfe848e
- testing · 2ede8235
  mohammad authored Jun 05, 2020
  
  2ede8235
06 May, 2020 4 commits
- addressed jareds comment · c9c69c1e
  mohammad authored May 06, 2020
  
  c9c69c1e
- tested and seems to be working · b1ac9fd3
  mohammad authored May 05, 2020
  
  b1ac9fd3
- added parameters sharing · f257d0ee
  Mohammad authored May 05, 2020
  
  f257d0ee
- added parameters sharing · 80f90dcd
  Mohammad authored May 05, 2020
  
  80f90dcd
21 Apr, 2020 1 commit
- removed bcast in local ddp · 41c1af0e
  Mohammad authored Apr 20, 2020
  
  41c1af0e
16 Apr, 2020 1 commit
- changed licence 2019 to 2020 · 463d1257
  Mohammad authored Apr 16, 2020
  
  463d1257
15 Apr, 2020 2 commits
- added faster L2 grad clipping and new torch gelu · 3ea5491e
  Mohammad authored Apr 14, 2020
  
  3ea5491e
- changed gpt2 masking to binary and masked_fill · 8ceed7c7
  Mohammad authored Apr 14, 2020
  
  8ceed7c7
14 Apr, 2020 1 commit
- Lint whole repo · b7f1b050
  Neel Kant authored Apr 14, 2020
  
  b7f1b050
10 Apr, 2020 1 commit
- fp32 fixes · 5f174c07
  Mohammad authored Apr 10, 2020
  
  5f174c07
03 Apr, 2020 1 commit
- working on refactoring text generation · ce29d4d5
  Mohammad authored Apr 02, 2020
  
  ce29d4d5
31 Mar, 2020 1 commit
- arguments in the model refactored · deffcb6a
  Mohammad authored Mar 31, 2020
  
  deffcb6a
29 Mar, 2020 1 commit
- utils partially refactored and tested · 3f58649b
  Mohammad authored Mar 28, 2020
  
  3f58649b
24 Mar, 2020 1 commit
- BERT downstream tasks: RACE, MNLI, QQP · 90ef2e28
  Mohammad Shoeybi authored Mar 23, 2020
  
  90ef2e28
10 Feb, 2020 1 commit
- Model parallel merger · 57c2060f
  Mohammad Shoeybi authored Feb 10, 2020
  
  57c2060f
14 Jan, 2020 2 commits
- added query-key layer scaling and softmax fp32 option · 6c2d0337
  Mohammad Shoeybi authored Jan 13, 2020
  
  6c2d0337
- added query-key layer scaling and softmax fp32 option · 691747b1
  Mohammad Shoeybi authored Jan 13, 2020
  
  691747b1
21 Nov, 2019 1 commit
- fixed padding issue · f6a6811f
  Mohammad Shoeybi authored Nov 20, 2019
  
  f6a6811f
29 Oct, 2019 1 commit
- Major refactoring, combining gpt2 and bert · 73af1290
  Mohammad Shoeybi authored Oct 29, 2019
  
  73af1290
08 Oct, 2019 1 commit
- created megatron package · b886b7bb
  Mohammad Shoeybi authored Oct 08, 2019
  
  b886b7bb