Commits · 56e16cbaf69f92da94a3a4bc9d5f9629f4634a5f · OpenDAS / Megatron-LM

30 Sep, 2020 1 commit
- checkpoint versioning · a8d47812
  Vijay Korthikanti authored Sep 29, 2020
  
  a8d47812
29 Sep, 2020 2 commits
- bug fix · 7d4ad51e
  Vijay Korthikanti authored Sep 29, 2020
  
  7d4ad51e
- support for old checkpoint format · 37ae6646
  Vijay Korthikanti authored Sep 28, 2020
  
  37ae6646
18 Sep, 2020 1 commit
- scaling-generalized masking-softmax fusion kernel · d73def35
  root authored Sep 18, 2020
  
  d73def35
11 Sep, 2020 1 commit
- Various speed optimizations. · 51a2e6b0
  Vijay Korthikanti authored Sep 11, 2020
  
  51a2e6b0
02 Sep, 2020 1 commit
- added splitting checkpointed activations · d6c4248b
  mohammad authored Sep 02, 2020
  
  d6c4248b
06 Aug, 2020 1 commit
- Implementing lazy parallel initialization · 5c04ceb3
  Boris Fomitchev authored Aug 05, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  5c04ceb3
29 Jul, 2020 1 commit
- Restored @script for openai_gelu · bb7c638f
  Boris Fomitchev authored Jul 29, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  bb7c638f
28 Jul, 2020 1 commit
- Replacing --erf-gelu option with explicit --onnx-safe option · fe794c5a
  Boris Fomitchev authored Jul 27, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  fe794c5a
26 Jul, 2020 2 commits
- fixing copy-paste · 7917774a
  Boris Fomitchev authored Jul 25, 2020
  
  7917774a
- Complete fix for APEX absence in NeMo · aa0ee72e
  Boris Fomitchev authored Jul 25, 2020
  
  aa0ee72e
24 Jul, 2020 3 commits
- Fixed APEX test · 8f0af7cd
  Boris Fomitchev authored Jul 24, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  8f0af7cd
- Warning added · b04eb0a2
  Boris Fomitchev authored Jul 23, 2020
  
  b04eb0a2
- ONNX export fix, including one for gelu() · 158a99c3
  Boris Fomitchev authored Jul 23, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  158a99c3
23 Jul, 2020 1 commit
- Address comments from last week · 11f76cd3
  Neel Kant authored Jul 22, 2020
  
  11f76cd3
22 Jul, 2020 1 commit
- Initialization fixes: allowing simple case like pytest pass, also making apex optional · 9026b86d
  Boris Fomitchev authored Jul 22, 2020
```
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
```
  9026b86d
07 Jul, 2020 1 commit
- Addressed Jared's comments · 8d7f508a
  Neel Kant authored Jul 06, 2020
  
  8d7f508a
29 Jun, 2020 1 commit
- Create tensors on cuda rather than copying · 76928caa
  Neel Kant authored Jun 29, 2020
  
  76928caa
24 Jun, 2020 1 commit
- More ict_merge changes and interactive testing · 3354081f
  Neel Kant authored Jun 23, 2020
  
  3354081f
22 Jun, 2020 1 commit
- Address most of comments from code review w/ Mohammad · 6495927e
  Neel Kant authored Jun 22, 2020
  
  6495927e
10 Jun, 2020 1 commit
- Ran and cleaned up · fcc500d6
  Neel Kant authored Jun 10, 2020
  
  fcc500d6
09 Jun, 2020 1 commit
- Correct retrieval utility and add salient span preprocessing · bf599e86
  Neel Kant authored Jun 08, 2020
  
  bf599e86
05 Jun, 2020 8 commits
- addressed jareds comments · 197c132e
  mohammad authored Jun 05, 2020
  
  197c132e
- added fp16 lm cross entropy to bert · 78022005
  mohammad authored Jun 05, 2020
  
  78022005
- added fp16 cross entropy loss option for gpt2 · 22e3c7e6
  mohammad authored Jun 05, 2020
  
  22e3c7e6
- added fp16 cross entropy loss option for gpt2 · acfe848e
  mohammad authored Jun 05, 2020
  
  acfe848e
- Some changes noticed late · c044f59a
  Neel Kant authored Jun 05, 2020
  
  c044f59a
- Prune changes to only be related to ICT · 32bb4edc
  Neel Kant authored Jun 05, 2020
  
  32bb4edc
- runs with new log loss but plateaus early · 20895f2c
  Neel Kant authored Jun 04, 2020
  
  20895f2c
- testing · 2ede8235
  mohammad authored Jun 05, 2020
  
  2ede8235
31 May, 2020 1 commit
- Misc changes · 51204a4d
  Neel Kant authored May 30, 2020
  
  51204a4d
27 May, 2020 1 commit
- Correct CrossEntropyLoss · 0e8f4331
  Neel Kant authored May 27, 2020
  
  0e8f4331
26 May, 2020 1 commit
- Corrected realm example building, misc improvements for async concurrency · 2fd4ea6c
  Neel Kant authored May 25, 2020
  
  2fd4ea6c
24 May, 2020 1 commit
- Fix token alignment, add mpu checkpointing, misc training code · 8e22824e
  Neel Kant authored May 24, 2020
  
  8e22824e
20 May, 2020 1 commit
- Async works for total 8 GPU, indexer debug mode · a670b6c9
  Neel Kant authored May 19, 2020
  
  a670b6c9
19 May, 2020 1 commit
- Full cycle of communication complete. Also added BasicIndexBuilder · 5684f904
  Neel Kant authored May 19, 2020
  
  5684f904
14 May, 2020 2 commits
- Add retrieval utility and autoresume for indexer · 2f7d666c
  Neel Kant authored May 14, 2020
  
  2f7d666c
- Minor adjustments to fit QA codebase · 9b9b8e01
  Neel Kant authored May 14, 2020
  
  9b9b8e01
12 May, 2020 1 commit
- faiss use_gpu · 6e256445
  Neel Kant authored May 12, 2020
  
  6e256445
07 May, 2020 1 commit
- Add primitive filesystem-based IPC for indexer and trainer jobs · 15d0d55b
  Neel Kant authored May 07, 2020
  
  15d0d55b