Commits · b1714c14ce24ed045d6b0e0d85a4e8744e2e24a7 · OpenDAS / Megatron-LM · GitLab

"vscode:/vscode.git/clone" did not exist on "5d071bb62c8960a68696c85730b437824a351880"

09 Nov, 2019 2 commits
- Skip any empty sentences during preprocessing. · 3f4bc91b
  Jared Casper authored Nov 08, 2019
  
  3f4bc91b
- Add ability to take a contiguous slice to indexed_dataset. · 61697cab
  Jared Casper authored Nov 08, 2019
```
Also removed the fix_lua_indexing stuff from fairseq.
```
  61697cab
08 Nov, 2019 1 commit
- Add document index to index file. An empty sentence no longer separate documents. · 87bbe9be
  Jared Casper authored Nov 07, 2019
  
  87bbe9be
07 Nov, 2019 1 commit
- Initial commit of multiprocess preprocess and extracted copy of fairseq's indexed_dataset. · 1237533e
  Jared Casper authored Nov 07, 2019
  
  1237533e