Commits · 7a3b4c158dbb63d9a1f5b18323b67bc10a2e67a6 · OpenDAS / Megatron-LM

14 Jan, 2020 1 commit
- exponential learning rate decay added · 7a3b4c15
  Mohammad Shoeybi authored Jan 13, 2020
  
  7a3b4c15
06 Jan, 2020 1 commit
- Merge branch 'mo_albert_data_loader' into 'master' · 2d76d065
  Jared Casper authored Jan 06, 2020
```
New data loader

See merge request ADLR/megatron-lm!16
```
  2d76d065
27 Dec, 2019 1 commit
- checked and bert, gpt, and albert albert run · f86bb671
  Mohammad Shoeybi authored Dec 26, 2019
  
  f86bb671
26 Dec, 2019 3 commits
- fixed gpt-2 dataloder · d6485684
  Mohammad Shoeybi authored Dec 26, 2019
  
  d6485684
- reverted init in data_utils · b8e0129f
  Mohammad Shoeybi authored Dec 26, 2019
  
  b8e0129f
- changed long seq length to 512 · 43dd21f7
  Mohammad Shoeybi authored Dec 26, 2019
  
  43dd21f7
25 Dec, 2019 1 commit
- added filtering based on sentence length · 2c58c9b0
  Mohammad Shoeybi authored Dec 24, 2019
  
  2c58c9b0
04 Dec, 2019 1 commit
- minor cleaning · 75a86a1d
  Mohammad Shoeybi authored Dec 03, 2019
  
  75a86a1d
02 Dec, 2019 1 commit
- Merge branch 'master' into mo_albert_data_loader · 5fc301aa
  Mohammad Shoeybi authored Dec 02, 2019
  
  5fc301aa
27 Nov, 2019 2 commits
- Merge branch 'refactor_gpt2' into 'master' · 1b290993
  Mostofa Patwary authored Nov 27, 2019
```
Refactor gpt2

See merge request ADLR/megatron-lm!15
```
  1b290993
- refactoring · f223ff67
  Mohammad Shoeybi authored Nov 27, 2019
  
  f223ff67
24 Nov, 2019 4 commits
- removed split dataset · 8179ebd3
  Mohammad Shoeybi authored Nov 24, 2019
  
  8179ebd3
- minor fix · 5300c69f
  Mohammad Shoeybi authored Nov 24, 2019
  
  5300c69f
- resolved reproducibity issue · e2add0fd
  Mohammad Shoeybi authored Nov 24, 2019
  
  e2add0fd
- zero worker seems to be working · 0601702a
  Mohammad Shoeybi authored Nov 24, 2019
  
  0601702a
21 Nov, 2019 1 commit
- fixed padding issue · f6a6811f
  Mohammad Shoeybi authored Nov 20, 2019
  
  f6a6811f
20 Nov, 2019 2 commits
- added write option for index map · 9ea9d50f
  Mohammad Shoeybi authored Nov 20, 2019
  
  9ea9d50f
- fixed couple of bugs in helpers · 690291dd
  Mohammad Shoeybi authored Nov 20, 2019
  
  690291dd
19 Nov, 2019 1 commit
- Plumbing for new albert dataset, including mods to arguments for data loaders. · 1a1cfcff
  Jared Casper authored Nov 18, 2019
  
  1a1cfcff
13 Nov, 2019 1 commit
- Plugging new dataset stuff together. · 0058b1a5
  Jared Casper authored Nov 13, 2019
```
Some cleanup of types, etc. in the helpers cpp code.
```
  0058b1a5
11 Nov, 2019 2 commits
- Merge branch 'albert_data_loader' of... · b1714c14
  Mohammad Shoeybi authored Nov 11, 2019
```
Merge branch 'albert_data_loader' of ssh://gitlab-master.nvidia.com:12051/ADLR/megatron-lm into albert_data_loader
```
  b1714c14
- c++ code working · f51ceb7c
  Mohammad Shoeybi authored Nov 10, 2019
  
  f51ceb7c
09 Nov, 2019 2 commits
- Skip any empty sentences during preprocessing. · 3f4bc91b
  Jared Casper authored Nov 08, 2019
  
  3f4bc91b
- Add ability to take a contiguous slice to indexed_dataset. · 61697cab
  Jared Casper authored Nov 08, 2019
```
Also removed the fix_lua_indexing stuff from fairseq.
```
  61697cab
08 Nov, 2019 5 commits
- before optimization · 6140718f
  Mohammad Shoeybi authored Nov 08, 2019
  
  6140718f
- built simple test for dataset · c125d247
  Mohammad Shoeybi authored Nov 07, 2019
  
  c125d247
- added dataset · 7120e931
  Mohammad Shoeybi authored Nov 07, 2019
  
  7120e931
- added training sample builder · adec01d0
  Mohammad Shoeybi authored Nov 07, 2019
  
  adec01d0
- Add document index to index file. An empty sentence no longer separate documents. · 87bbe9be
  Jared Casper authored Nov 07, 2019
  
  87bbe9be
07 Nov, 2019 2 commits
- Initial commit of multiprocess preprocess and extracted copy of fairseq's indexed_dataset. · 1237533e
  Jared Casper authored Nov 07, 2019
  
  1237533e
- added bert tokenization · 0ceeb3b4
  Mohammad Shoeybi authored Nov 06, 2019
  
  0ceeb3b4
04 Nov, 2019 2 commits
- Merge branch 'multiple-eos-edge-case' into 'master' · 0f873f97
  Mohammad Shoeybi authored Nov 04, 2019
```
Fixed edge case with multiple end of sequence in one sequence

See merge request ADLR/megatron-lm!12
```
  0f873f97
- Fixed edge case with multiple end of sequence in one sequence · 872b4a6a
  Raul Puri authored Nov 04, 2019
  
  872b4a6a
01 Nov, 2019 1 commit
- Merge branch 'code_reuse' into 'master' · cf2f4d9d
  Jared Casper authored Oct 31, 2019
```
refactored for code reuse

See merge request ADLR/megatron-lm!11
```
  cf2f4d9d
31 Oct, 2019 2 commits
- refactored for code reuse · cbd8c054
  Mohammad Shoeybi authored Oct 30, 2019
  
  cbd8c054
- refacotred for code reuse · 34be7dd3
  Mohammad Shoeybi authored Oct 30, 2019
  
  34be7dd3
29 Oct, 2019 2 commits
- Merge branch 'transformer_refactoring_from_pretrain_refactoring' into 'master' · beb3e0d3
  Raul Puri authored Oct 29, 2019
```
Major refactoring, combining gpt2 and bert

See merge request ADLR/megatron-lm!8
```
  beb3e0d3
- Major refactoring, combining gpt2 and bert · 73af1290
  Mohammad Shoeybi authored Oct 29, 2019
  
  73af1290
09 Oct, 2019 2 commits
- Merge branch 'move_vocab_padding_to_utils' into 'master' · ebbe40cd
  Raul Puri authored Oct 09, 2019
```
Move vocab padding to utils

See merge request ADLR/megatron-lm!6
```
  ebbe40cd
- Merge branch 'master' into move_vocab_padding_to_utils · 7e46ca58
  Mohammad Shoeybi authored Oct 09, 2019
  
  7e46ca58