Commits · 1a1cfcff338acac65aa60e70348adb124f8e631f · OpenDAS / Megatron-LM

"llm/llama.cpp/ggml/src/ggml-sycl.cpp" did not exist on "b0135f4b9b176eab9155b660d04c9ca2a1ec2341"

19 Nov, 2019 1 commit
- Plumbing for new albert dataset, including mods to arguments for data loaders. · 1a1cfcff
  Jared Casper authored Nov 18, 2019
  
  1a1cfcff
13 Nov, 2019 1 commit
- Plugging new dataset stuff together. · 0058b1a5
  Jared Casper authored Nov 13, 2019
```
Some cleanup of types, etc. in the helpers cpp code.
```
  0058b1a5
11 Nov, 2019 1 commit
- c++ code working · f51ceb7c
  Mohammad Shoeybi authored Nov 10, 2019
  
  f51ceb7c
09 Nov, 2019 2 commits
- Skip any empty sentences during preprocessing. · 3f4bc91b
  Jared Casper authored Nov 08, 2019
  
  3f4bc91b
- Add ability to take a contiguous slice to indexed_dataset. · 61697cab
  Jared Casper authored Nov 08, 2019
```
Also removed the fix_lua_indexing stuff from fairseq.
```
  61697cab
08 Nov, 2019 5 commits
- before optimization · 6140718f
  Mohammad Shoeybi authored Nov 08, 2019
  
  6140718f
- built simple test for dataset · c125d247
  Mohammad Shoeybi authored Nov 07, 2019
  
  c125d247
- added dataset · 7120e931
  Mohammad Shoeybi authored Nov 07, 2019
  
  7120e931
- added training sample builder · adec01d0
  Mohammad Shoeybi authored Nov 07, 2019
  
  adec01d0
- Add document index to index file. An empty sentence no longer separate documents. · 87bbe9be
  Jared Casper authored Nov 07, 2019
  
  87bbe9be
07 Nov, 2019 2 commits
- Initial commit of multiprocess preprocess and extracted copy of fairseq's indexed_dataset. · 1237533e
  Jared Casper authored Nov 07, 2019
  
  1237533e
- added bert tokenization · 0ceeb3b4
  Mohammad Shoeybi authored Nov 06, 2019
  
  0ceeb3b4