Commits · 3f58649b794b111caf90d8d00a0024677922bf2a · OpenDAS / Megatron-LM

29 Mar, 2020 1 commit
- utils partially refactored and tested · 3f58649b
  Mohammad authored Mar 28, 2020
  
  3f58649b
24 Nov, 2019 1 commit
- zero worker seems to be working · 0601702a
  Mohammad Shoeybi authored Nov 24, 2019
  
  0601702a
20 Nov, 2019 1 commit
- fixed couple of bugs in helpers · 690291dd
  Mohammad Shoeybi authored Nov 20, 2019
  
  690291dd
19 Nov, 2019 1 commit
- Plumbing for new albert dataset, including mods to arguments for data loaders. · 1a1cfcff
  Jared Casper authored Nov 18, 2019
  
  1a1cfcff
13 Nov, 2019 1 commit
- Plugging new dataset stuff together. · 0058b1a5
  Jared Casper authored Nov 13, 2019
```
Some cleanup of types, etc. in the helpers cpp code.
```
  0058b1a5
09 Nov, 2019 1 commit
- Add ability to take a contiguous slice to indexed_dataset. · 61697cab
  Jared Casper authored Nov 08, 2019
```
Also removed the fix_lua_indexing stuff from fairseq.
```
  61697cab
08 Nov, 2019 1 commit
- Add document index to index file. An empty sentence no longer separate documents. · 87bbe9be
  Jared Casper authored Nov 07, 2019
  
  87bbe9be
07 Nov, 2019 1 commit
- Initial commit of multiprocess preprocess and extracted copy of fairseq's indexed_dataset. · 1237533e
  Jared Casper authored Nov 07, 2019
  
  1237533e