Commits · 050c24d52c100b95dacc8dc437060953b3b664f1 · OpenDAS / Megatron-LM

"vscode:/vscode.git/clone" did not exist on "a578d300ba036221894113c7ec0d415a8dbb319e"

16 Apr, 2020 17 commits
- Merge branch 'staging_docs' into 'staging' · 050c24d5
  Steven Steinke authored Apr 16, 2020
```
Staging docs

See merge request ADLR/megatron-lm!64
```
  050c24d5
- looks great · 3df23c72
  Mohammad authored Apr 16, 2020
  
  3df23c72
- latest · 0d8f1243
  Mohammad authored Apr 16, 2020
  
  0d8f1243
- latest · 1f45adcc
  Mohammad authored Apr 16, 2020
  
  1f45adcc
- updated scripts · ca19a8b5
  Mohammad authored Apr 16, 2020
  
  ca19a8b5
- made monir changes to the first two paras · fe40745e
  Mohammad authored Apr 16, 2020
  
  fe40745e
- Merge branch 'staging' into staging_docs · add35c17
  Mohammad authored Apr 16, 2020
  
  add35c17
- Merge branch 'staging_dataset_error_reporting' into 'staging' · d928dd51
  Mohammad Shoeybi authored Apr 16, 2020
```
Better error reporting when a dataset doesn't exist.

See merge request ADLR/megatron-lm!63
```
  d928dd51
- moved steves branch · 78cb1781
  Mohammad authored Apr 16, 2020
  
  78cb1781
- Merge branch 'staging_fix_task_main_imports' into 'staging' · c6b5c137
  Raul Puri authored Apr 16, 2020
```
Staging fix task main imports

See merge request ADLR/megatron-lm!62
```
  c6b5c137
- Better error reporting when a dataset doesn't exist. · f2779b18
  Jared Casper authored Apr 16, 2020
  
  f2779b18
- fixed tasks/main imports · 4d39aba4
  Mohammad authored Apr 16, 2020
  
  4d39aba4
- Merge branch 'staging_compile_helpers' into 'staging' · f1f9fa0a
  Jared Casper authored Apr 16, 2020
```
runtime compilation of helpers

See merge request ADLR/megatron-lm!61
```
  f1f9fa0a
- Check that the dataset helpers make runs successfully. · 9073c4bd
  Jared Casper authored Apr 16, 2020
  
  9073c4bd
- added runtime compilation for helpers · c6882114
  Mohammad authored Apr 15, 2020
  
  c6882114
- Merge branch 'staging_optimization_level_2' into 'staging' · 8134313d
  Jared Casper authored Apr 15, 2020
```
Staging optimization level 2

See merge request ADLR/megatron-lm!55
```
  8134313d
- Merge branch 'staging_optimization' into 'staging' · 97913a1f
  Jared Casper authored Apr 15, 2020
```
Staging optimization level 1

See merge request ADLR/megatron-lm!54
```
  97913a1f
15 Apr, 2020 11 commits
- addressed jareds comment · a4489ffb
  Mohammad authored Apr 15, 2020
  
  a4489ffb
- addressed jareds comments · b6e73188
  Mohammad authored Apr 15, 2020
  
  b6e73188
- added faster L2 grad clipping and new torch gelu · 3ea5491e
  Mohammad authored Apr 14, 2020
  
  3ea5491e
- added multi-tensor-apply to fp16 · 99410264
  Mohammad authored Apr 14, 2020
  
  99410264
- Merge branch 'staging' into staging_optimization · 2467ae15
  Mohammad authored Apr 14, 2020
  
  2467ae15
- Merge branch 'staging_optional_args' into 'staging' · be5300b7
  Neel Kant authored Apr 14, 2020
```
made size arguments optional so they can be set from input function

See merge request ADLR/megatron-lm!53
```
  be5300b7
- using None for resetting gradients · 3a701586
  Mohammad authored Apr 14, 2020
  
  3a701586
- addressed neels comments · 8e8e4548
  Mohammad authored Apr 14, 2020
  
  8e8e4548
- changed gpt2 masking to binary and masked_fill · 8ceed7c7
  Mohammad authored Apr 14, 2020
  
  8ceed7c7
- minor cleanup · c0a59a66
  Mohammad authored Apr 14, 2020
  
  c0a59a66
- made size arguments optional so they can be set from input function · 368ad0d3
  Mohammad authored Apr 14, 2020
  
  368ad0d3
14 Apr, 2020 11 commits
- Merge branch 'staging-lint' into 'staging' · 7d75b3b5
  Mohammad Shoeybi authored Apr 14, 2020
```
Lint whole repo

See merge request ADLR/megatron-lm!51
```
  7d75b3b5
- Lint whole repo · b7f1b050
  Neel Kant authored Apr 14, 2020
  
  b7f1b050
- Merge branch 'move_openwebtext' into 'staging' · c99fa80c
  Mohammad Shoeybi authored Apr 14, 2020
```
Move openwebtext scripts to tools directory.

See merge request ADLR/megatron-lm!50
```
  c99fa80c
- Move openwebtext scripts to tools directory. · 0dc10d09
  Jared Casper authored Apr 14, 2020
  
  0dc10d09
- Merge branch 'staging_gpt2_dataloader' into 'staging' · 69d8ff1c
  Jared Casper authored Apr 14, 2020
```
GPT2 dataloader using indexed dataset

See merge request ADLR/megatron-lm!45
```
  69d8ff1c
- removed docker directory · 0afe1fde
  Mohammad authored Apr 14, 2020
  
  0afe1fde
- Merge branch 'staging' into staging_gpt2_dataloader · 6ae4a03c
  Mohammad authored Apr 14, 2020
  
  6ae4a03c
- Merge branch 'staging_distributed_cpu_fix' into 'staging' · 7890681a
  Mohammad Shoeybi authored Apr 14, 2020
```
Added logic for initilialize_megatron to work with cpu only jobs. This is...

See merge request ADLR/megatron-lm!46
```
  7890681a
- Merge branch 'staging_mp_merge' into 'staging_gpt2_dataloader' · e06159f2
  Jared Casper authored Apr 14, 2020
```
refactored merge_mp_partitions.py

See merge request ADLR/megatron-lm!47
```
  e06159f2
- Merge branch 'indexed_dataset_nocast' into 'staging_gpt2_dataloader' · 76e9a71e
  Mohammad Shoeybi authored Apr 13, 2020
```
Do not cast return type of indexed dataset

See merge request ADLR/megatron-lm!48
```
  76e9a71e
- added comment to build_index_mappings · d8bb57ad
  Mohammad authored Apr 13, 2020
  
  d8bb57ad
13 Apr, 2020 1 commit
- Do not cast data returned from indexed_dataset to int64, rely on caller to... · 71cf1866
  Jared Casper authored Apr 13, 2020
```
Do not cast data returned from indexed_dataset to int64, rely on caller to cast to appropriate type.
```
  71cf1866