Commits · ff12df6bfb73e9f412b6188497d6e4eeef00ce61 · OpenDAS / Megatron-LM

30 Nov, 2020 1 commit
- refactored learning rate scheduler so addition of variable batch size is easier · ff12df6b
  mohammad authored Nov 29, 2020
  
  ff12df6b
28 Nov, 2020 1 commit
- added consumed tokens to checkpoints and some refactoring · f0a445fa
  mohammad authored Nov 27, 2020
  
  f0a445fa
26 Nov, 2020 1 commit
- simplified sampler · 4311b695
  mohammad authored Nov 25, 2020
  
  4311b695
03 Nov, 2020 1 commit
- fixed loss average when all but one value is skipped · 664cd28b
  mohammad authored Nov 03, 2020
  
  664cd28b
14 Oct, 2020 1 commit
- added adam betas and eps as arguments · 48269d8d
  Mostofa Patwary authored Oct 13, 2020
  
  48269d8d
25 Sep, 2020 1 commit
- fixed the interation count · 2db01991
  mohammad authored Sep 24, 2020
  
  2db01991
24 Sep, 2020 1 commit
- nan values report added · abd7636e
  mohammad authored Sep 23, 2020
  
  abd7636e
22 Sep, 2020 1 commit
- fixed logging bug · a74f8d9c
  mohammad authored Sep 22, 2020
  
  a74f8d9c
11 Sep, 2020 1 commit
- Various speed optimizations. · 51a2e6b0
  Vijay Korthikanti authored Sep 11, 2020
  
  51a2e6b0
14 Jul, 2020 1 commit
- Additional refactoring · b9bd1a11
  Neel Kant authored Jul 13, 2020
  
  b9bd1a11
07 Jul, 2020 1 commit
- Addressed Jared's comments · 8d7f508a
  Neel Kant authored Jul 06, 2020
  
  8d7f508a
25 Jun, 2020 1 commit
- Add stanza NER salient span masking · e7045139
  Neel Kant authored Jun 24, 2020
  
  e7045139
24 Jun, 2020 1 commit
- More ict_merge changes and interactive testing · 3354081f
  Neel Kant authored Jun 23, 2020
  
  3354081f
22 Jun, 2020 1 commit
- Address most of comments from code review w/ Mohammad · 6495927e
  Neel Kant authored Jun 22, 2020
  
  6495927e
15 Jun, 2020 1 commit
- Debugging done on Circe · b8bb0b49
  Mohammad authored Jun 15, 2020
  
  b8bb0b49
09 Jun, 2020 1 commit
- Correct retrieval utility and add salient span preprocessing · bf599e86
  Neel Kant authored Jun 08, 2020
  
  bf599e86
05 Jun, 2020 4 commits
- added fp16 cross entropy loss option for gpt2 · acfe848e
  mohammad authored Jun 05, 2020
  
  acfe848e
- Prune changes to only be related to ICT · 32bb4edc
  Neel Kant authored Jun 05, 2020
  
  32bb4edc
- runs with new log loss but plateaus early · 20895f2c
  Neel Kant authored Jun 04, 2020
  
  20895f2c
- testing · 2ede8235
  mohammad authored Jun 05, 2020
  
  2ede8235
31 May, 2020 1 commit
- Misc changes · 51204a4d
  Neel Kant authored May 30, 2020
  
  51204a4d
27 May, 2020 2 commits
- Correct indexer seq length · dfb907fe
  Neel Kant authored May 27, 2020
  
  dfb907fe
- Correct CrossEntropyLoss · 0e8f4331
  Neel Kant authored May 27, 2020
  
  0e8f4331
26 May, 2020 1 commit
- Corrected realm example building, misc improvements for async concurrency · 2fd4ea6c
  Neel Kant authored May 25, 2020
  
  2fd4ea6c
24 May, 2020 1 commit
- Fix token alignment, add mpu checkpointing, misc training code · 8e22824e
  Neel Kant authored May 24, 2020
  
  8e22824e
21 May, 2020 1 commit
- Change sync variable to gloo backend · 05ea0cca
  Neel Kant authored May 21, 2020
  
  05ea0cca
20 May, 2020 1 commit
- Async works for total 8 GPU, indexer debug mode · a670b6c9
  Neel Kant authored May 19, 2020
  
  a670b6c9
19 May, 2020 2 commits
- Full cycle of communication complete. Also added BasicIndexBuilder · 5684f904
  Neel Kant authored May 19, 2020
  
  5684f904
- Reorganize indexer. Things run up to saving model checkpoint and repeating · d4b00be0
  Neel Kant authored May 19, 2020
  
  d4b00be0
12 May, 2020 1 commit
- Debug and run salient span masking · c150c48b
  Neel Kant authored May 12, 2020
  
  c150c48b
07 May, 2020 1 commit
- Add primitive filesystem-based IPC for indexer and trainer jobs · 15d0d55b
  Neel Kant authored May 07, 2020
  
  15d0d55b
29 Apr, 2020 1 commit
- Update and test misc functionality · 6c0a5bd8
  Neel Kant authored Apr 28, 2020
  
  6c0a5bd8
24 Apr, 2020 1 commit
- Mostly debugged realm-mlm · d7022c72
  Neel Kant authored Apr 24, 2020
  
  d7022c72
17 Apr, 2020 1 commit
- Refactor before merge request · 4abd7ce2
  Neel Kant authored Apr 17, 2020
  
  4abd7ce2
16 Apr, 2020 1 commit
- changed licence 2019 to 2020 · 463d1257
  Mohammad authored Apr 16, 2020
  
  463d1257
15 Apr, 2020 1 commit
- using None for resetting gradients · 3a701586
  Mohammad authored Apr 14, 2020
  
  3a701586
14 Apr, 2020 1 commit
- Lint whole repo · b7f1b050
  Neel Kant authored Apr 14, 2020
  
  b7f1b050
13 Apr, 2020 1 commit
- data loading for BERT and GPT cleaned up · 2e38461b
  Mohammad authored Apr 12, 2020
  
  2e38461b
10 Apr, 2020 1 commit
- fp32 fixes · 5f174c07
  Mohammad authored Apr 10, 2020
  
  5f174c07
07 Apr, 2020 1 commit
- Fix issue with validation dataloader · 064bdc46
  Neel Kant authored Apr 07, 2020
  
  064bdc46