Commits · 87bbe9be714ce101db5a0a9c759a286414375dfa · OpenDAS / Megatron-LM

08 Nov, 2019 1 commit
- Add document index to index file. An empty sentence no longer separate documents. · 87bbe9be
  Jared Casper authored Nov 07, 2019
  
  87bbe9be
07 Nov, 2019 2 commits
- Initial commit of multiprocess preprocess and extracted copy of fairseq's indexed_dataset. · 1237533e
  Jared Casper authored Nov 07, 2019
  
  1237533e
- added bert tokenization · 0ceeb3b4
  Mohammad Shoeybi authored Nov 06, 2019
  
  0ceeb3b4
31 Oct, 2019 2 commits
- refactored for code reuse · cbd8c054
  Mohammad Shoeybi authored Oct 30, 2019
  
  cbd8c054
- refacotred for code reuse · 34be7dd3
  Mohammad Shoeybi authored Oct 30, 2019
  
  34be7dd3
29 Oct, 2019 1 commit
- Major refactoring, combining gpt2 and bert · 73af1290
  Mohammad Shoeybi authored Oct 29, 2019
  
  73af1290
09 Oct, 2019 2 commits
- moved padding to utils · 2e6d5ed9
  Mohammad Shoeybi authored Oct 09, 2019
  
  2e6d5ed9
- fixed deserializing issue with old checkpoint · ee38e7f9
  Mohammad Shoeybi authored Oct 09, 2019
  
  ee38e7f9
08 Oct, 2019 2 commits
- moved few common elements between bert and gpt to utils · 787c1a0b
  Mohammad Shoeybi authored Oct 08, 2019
  
  787c1a0b
- created megatron package · b886b7bb
  Mohammad Shoeybi authored Oct 08, 2019
  
  b886b7bb