Commits · f51ceb7c9b6d01e36b20d8aa4cb6be139fa70180 · OpenDAS / Megatron-LM

11 Nov, 2019 1 commit
- c++ code working · f51ceb7c
  Mohammad Shoeybi authored Nov 10, 2019
  
  f51ceb7c
08 Nov, 2019 5 commits
- before optimization · 6140718f
  Mohammad Shoeybi authored Nov 08, 2019
  
  6140718f
- built simple test for dataset · c125d247
  Mohammad Shoeybi authored Nov 07, 2019
  
  c125d247
- added dataset · 7120e931
  Mohammad Shoeybi authored Nov 07, 2019
  
  7120e931
- added training sample builder · adec01d0
  Mohammad Shoeybi authored Nov 07, 2019
  
  adec01d0
- Add document index to index file. An empty sentence no longer separate documents. · 87bbe9be
  Jared Casper authored Nov 07, 2019
  
  87bbe9be
07 Nov, 2019 2 commits
- Initial commit of multiprocess preprocess and extracted copy of fairseq's indexed_dataset. · 1237533e
  Jared Casper authored Nov 07, 2019
  
  1237533e
- added bert tokenization · 0ceeb3b4
  Mohammad Shoeybi authored Nov 06, 2019
  
  0ceeb3b4