Commits · 9b174da873cf8e84f1bef1b6a88d98ff7791946e · OpenDAS / Megatron-LM

"git@developer.sourcefind.cn:Fzc7075/nunchaku.git" did not exist on "bf4adfeb771b2be8e91d270e2da064b7c1f5145f"

12 Nov, 2020 2 commits

Intra-layer MP -> Tensor MP, Inter-layer MP -> Pipeline MP · 52a5f2f2
Deepak Narayanan authored Oct 20, 2020

52a5f2f2

Pipeline parallelism implementation with periodic full-pipeline syncs · 7abd3e90

Deepak Narayanan authored Aug 29, 2020

Also includes following changes for inter-layer model-parallel implementation:
- Refactoring of model implementations
- Training loop changes to support inter-layer communication using `ring_exchange`
- New groups for inter-layer communication
- Checkpoint changes
- Command line arguments

7abd3e90

09 Jun, 2020 1 commit
- Correct retrieval utility and add salient span preprocessing · bf599e86
  Neel Kant authored Jun 08, 2020
  
  bf599e86
24 Apr, 2020 1 commit
- added case wordpiece arguments · ef5b2f06
  Raul Puri authored Apr 24, 2020
  
  ef5b2f06
16 Apr, 2020 1 commit
- changed licence 2019 to 2020 · 463d1257
  Mohammad authored Apr 16, 2020
  
  463d1257
13 Apr, 2020 1 commit
- refactored merge_mp_partitions.py · 03d0d93d
  Mohammad authored Apr 12, 2020
  
  03d0d93d
08 Apr, 2020 3 commits
- Add sentence or document suffix to output filename of preprocess_data.py · 1e283449
  Jared Casper authored Apr 08, 2020
  
  1e283449
- New tools directory. · 3500d036
  Jared Casper authored Apr 08, 2020
  
  3500d036
- Updates to preprocess_data.py and indexed_dataset. · da0562fc
  Jared Casper authored Apr 07, 2020
```
preprocess_data:
- Adds ability to not split sentences. This is used for gpt2 datasets.

- Adds ability to create multiple datasets from different json keys,
this is current untested.

indexed_dataset:
- Add new "get" function to get a portion of an entry.
```
  da0562fc