Commits · a0bea42531bae13f9a4aa3e2c08c1aa30a66c6c5 · OpenDAS / Megatron-LM

03 Feb, 2022 1 commit
- Removing unnecessary permutes and scatter/gather · a0bea425
  rprenger authored Feb 03, 2022
  
  a0bea425
17 Dec, 2021 1 commit
- Cleaning up Switch Transformer code · be136206
  rprenger authored Dec 17, 2021
  
  be136206
10 Dec, 2021 2 commits
- Adding scripts for parameter vs. compute experiment · 1ec6e720
  rprenger authored Dec 10, 2021
  
  1ec6e720
- Faster Switch code · 4018d92c
  rprenger authored Dec 02, 2021
  
  4018d92c
30 Nov, 2021 2 commits
- Merge branch 'README-NeMo' into 'main' · d4169684
  Jared Casper authored Nov 30, 2021
```
Mention NeMo Megatron in README

See merge request ADLR/megatron-lm!366
```
  d4169684
- Mention NeMo Megatron in README · 84529b20
  Jared Casper authored Nov 30, 2021
  
  84529b20
24 Nov, 2021 3 commits
- Merge branch 'lmcafee/dst-chk-act' into 'main' · 60750922
  Jared Casper authored Nov 23, 2021
```
Fixed distributed checkpoint activations to work on all layers

See merge request ADLR/megatron-lm!365
```
  60750922
- Merge branch 'rescaling' into 'main' · 2ad00f4e
  Jared Casper authored Nov 23, 2021
```
Add support for signal-based dynamic checkpointing

See merge request ADLR/megatron-lm!361
```
  2ad00f4e
- Add support for signal-based dynamic checkpointing · dbe6c728
  Szymon Migacz authored Nov 23, 2021
  
  dbe6c728
23 Nov, 2021 1 commit
- removed assert that prevented virtual pipelinining to be used with distributed checkpointing · 1b28a51b
  Lawrence McAfee authored Nov 23, 2021
  
  1b28a51b
22 Nov, 2021 3 commits
- assert minimum pytorch version for distributed checkpoint activations · be0da070
  Lawrence McAfee authored Nov 22, 2021
  
  be0da070
- removed unused 'get_args' import · 941a793f
  Lawrence McAfee authored Nov 22, 2021
  
  941a793f
- removed distribute_checkpointed_activations_helper() · 5993f04b
  Lawrence McAfee authored Nov 22, 2021
  
  5993f04b
18 Nov, 2021 2 commits
- Merge branch 'slym/persist-ln-fix' into 'main' · d6380fda
  Jared Casper authored Nov 18, 2021
```
add a kernel import guard for persistent layer norm

See merge request ADLR/megatron-lm!363
```
  d6380fda
- add a kernel import guard for persistent layer norm · 03a2903f
  Sangkug Lym authored Nov 18, 2021
  
  03a2903f
17 Nov, 2021 1 commit
- Merge branch 'slym/persist_ln' into 'main' · 4a3213f1
  Jared Casper authored Nov 17, 2021
```
Persistent layer norm

See merge request ADLR/megatron-lm!351
```
  4a3213f1
11 Nov, 2021 1 commit

Sangkug Lym authored Oct 08, 2021

fix the guard to fall back to the baseline fused layer norm kernel

Persisten ln: move the guard for supported hidden sizes to layer norm module

a2fdcdf0

05 Nov, 2021 1 commit
- Merge branch 'finetune-fix' into 'main' · 3ae12a47
  Jared Casper authored Nov 04, 2021
```
Fix finetuning tasks after T5 pipeline merge.

See merge request ADLR/megatron-lm!343
```
  3ae12a47
29 Oct, 2021 2 commits
- Merge branch 'inference_context_memory' into 'main' · f5345dfa
  Jared Casper authored Oct 29, 2021
```
made model stateless with respect to inference

See merge request ADLR/megatron-lm!348
```
  f5345dfa
- made model stateless with respect to inference · efc750b6
  mshoeybi authored Oct 29, 2021
  
  efc750b6
25 Oct, 2021 1 commit

Merge branch 'just_score' into 'main' · d33460df

Jared Casper authored Oct 25, 2021

This adds a function for the case where the user only wants the log-probabilities "tokens_to_generate=0".

See merge request ADLR/megatron-lm!345

d33460df

22 Oct, 2021 1 commit
- Removing dead code · d7699436
  rprenger authored Oct 22, 2021
  
  d7699436
20 Oct, 2021 1 commit
- getting rid of unnecessary just_score · 5ab28fd1
  rprenger authored Oct 20, 2021
  
  5ab28fd1
18 Oct, 2021 3 commits
- Fixing logic in server accepting tokens_to_generate=0 · a3770921
  rprenger authored Oct 18, 2021
  
  a3770921
- Merge branch 'readme-clarity' into 'main' · 4145f091
  Mohammad Shoeybi authored Oct 18, 2021
```
Clarify README regarding benchmarks.

See merge request ADLR/megatron-lm!344
```
  4145f091
- Clarify README regarding benchmarks. · f482bc50
  Jared Casper authored Oct 18, 2021
  
  f482bc50
16 Oct, 2021 3 commits
- Fixing a bug in merge · 87d08f4b
  rprenger authored Oct 15, 2021
  
  87d08f4b
- cleaning up merge conflicts · 803ae5ee
  rprenger authored Oct 15, 2021
  
  803ae5ee
- Getting tokens_to_generate=0 to work · 9cc286ba
  rprenger authored Oct 15, 2021
  
  9cc286ba
15 Oct, 2021 5 commits
- Merge branch 'inference' into 'main' · 0b0e37f0
  Jared Casper authored Oct 15, 2021
```
Inference refactoring

See merge request ADLR/megatron-lm!339
```
  0b0e37f0
- Fix finetuning tasks after T5 pipeline merge. · a20445d3
  Jared Casper authored Oct 15, 2021
  
  a20445d3
- Allowing for a 0 tokens/just scoring mode · 21d2b0fe
  rprenger authored Oct 15, 2021
  
  21d2b0fe
- removed greedy argument · 71359e1f
  mshoeybi authored Oct 15, 2021
  
  71359e1f
- removed return all probs · c6e7c7fd
  mshoeybi authored Oct 15, 2021
  
  c6e7c7fd
14 Oct, 2021 1 commit
- Changing defaults and query sanitation to keep it from crashing on reasonable queries · 8d405805
  rprenger authored Oct 14, 2021
  
  8d405805
13 Oct, 2021 3 commits
- tested and working · 0dd5cc75
  mshoeybi authored Oct 13, 2021
  
  0dd5cc75
- changed inference to text generation · ee536123
  mshoeybi authored Oct 13, 2021
  
  ee536123
- addressed Jareds comments · 8126d777
  mshoeybi authored Oct 13, 2021
  
  8126d777
12 Oct, 2021 2 commits
- fixed a bug · b10ca211
  mshoeybi authored Oct 12, 2021
  
  b10ca211
- Merge branch 'main' into inference · 8f7c0d11
  mshoeybi authored Oct 12, 2021
  
  8f7c0d11