Commits · 4554c3fed9a5b7daa5f564c84c71b8c689ba4f02 · OpenDAS / Megatron-LM

11 Feb, 2022 1 commit
- adress review comments · 4554c3fe
  Vijay Korthikanti authored Feb 11, 2022
  
  4554c3fe
01 Feb, 2022 3 commits
- more minor fixes · 3f1a728a
  Vijay Korthikanti authored Feb 01, 2022
  
  3f1a728a
- minor fixes · e1f9c3a5
  Vijay Korthikanti authored Feb 01, 2022
  
  e1f9c3a5
- vision third phase merge: pretraining methods + mit,swin backbones · 01a82723
  Vijay Korthikanti authored Jan 31, 2022
  
  01a82723
29 Jan, 2022 1 commit
- typo fix · 2b628f96
  Vijay Korthikanti authored Jan 28, 2022
  
  2b628f96
28 Jan, 2022 2 commits
- more naming cleanup · 641408f5
  Vijay Korthikanti authored Jan 28, 2022
  
  641408f5
- changing class name AnnealingLR to OptimizerParamScheduler · 04ecc834
  Vijay Korthikanti authored Jan 28, 2022
  
  04ecc834
27 Jan, 2022 1 commit
- address review comments · 53931b8b
  Vijay Korthikanti authored Jan 27, 2022
  
  53931b8b
26 Jan, 2022 1 commit
- address review comments · 8acbbe25
  Vijay Korthikanti authored Jan 26, 2022
  
  8acbbe25
24 Jan, 2022 1 commit
- minor fixes · 7e810e41
  Vijay Korthikanti authored Jan 24, 2022
  
  7e810e41
22 Jan, 2022 1 commit
- second phase of vision code merge · d8c85650
  Vijay Korthikanti authored Jan 21, 2022
  
  d8c85650
20 Jan, 2022 1 commit
- address review comments · 798b6a64
  Vijay Korthikanti authored Jan 20, 2022
  
  798b6a64
19 Jan, 2022 2 commits
- using all_gather instead of gather (nccl does not support gather) · b037a69e
  Vijay Korthikanti authored Jan 19, 2022
  
  b037a69e
- flag for data parallel random initialization · a7ee77ea
  Vijay Korthikanti authored Jan 19, 2022
  
  a7ee77ea
15 Jan, 2022 2 commits
- minor fix · fd8dd9c0
  Vijay Korthikanti authored Jan 14, 2022
  
  fd8dd9c0
- checkpointing rng_state of all data parallel ranks · 09d38884
  Vijay Korthikanti authored Jan 14, 2022
  
  09d38884
14 Jan, 2022 1 commit
- address review comments · 343dc97a
  Vijay Korthikanti authored Jan 14, 2022
  
  343dc97a
13 Jan, 2022 1 commit
- minor fixes · 3b29f0e7
  Vijay Korthikanti authored Jan 13, 2022
  
  3b29f0e7
12 Jan, 2022 1 commit
- Phase1 merge: vit optimizations + dataset enhancements + scaled_softmax kernel · 7a77abd9
  Vijay Korthikanti authored Jan 12, 2022
  
  7a77abd9
04 Jan, 2022 2 commits
- added flag to control deallocation of pipeline outputs · de1ae5b2
  Lawrence McAfee authored Jan 04, 2022
  
  de1ae5b2
- update comment · d439b417
  Vijay Korthikanti authored Jan 04, 2022
  
  d439b417
31 Dec, 2021 1 commit
- simplify code · 9da6e975
  Vijay Korthikanti authored Dec 30, 2021
  
  9da6e975
30 Dec, 2021 2 commits
- avoid allocation of word embedding for head in T5 pp=2 case · ade99d61
  Vijay Korthikanti authored Dec 30, 2021
  
  ade99d61
- fixing the corner case pp=2 · 26ea8314
  Vijay Korthikanti authored Dec 30, 2021
  
  26ea8314
21 Dec, 2021 1 commit
- comments · 81ad628e
  Vijay Korthikanti authored Dec 21, 2021
  
  81ad628e
17 Dec, 2021 2 commits
- minor fixes · f2bf5a56
  Vijay Korthikanti authored Dec 17, 2021
  
  f2bf5a56
- pipeline_fixes · 17843605
  Vijay Korthikanti authored Dec 17, 2021
  
  17843605
10 Dec, 2021 1 commit
- output tensor now allocated directly on gpu. · 85876092
  Lawrence McAfee authored Dec 10, 2021
  
  85876092
08 Dec, 2021 1 commit
- added docstrings to new methods · e33d63cc
  Lawrence McAfee authored Dec 08, 2021
  
  e33d63cc
07 Dec, 2021 2 commits
- working: interleaving; free_output_tensor() now handles none/tensor/list · 2f25c570
  Lawrence McAfee authored Dec 07, 2021
  
  2f25c570
- working for pure pipeline parallelism, w/ no interleaving · 86da10e9
  Lawrence McAfee authored Dec 07, 2021
  
  86da10e9
02 Dec, 2021 2 commits
- update training.py · e1b3fc8c
  zihanl authored Dec 01, 2021
  
  e1b3fc8c
- update tokenizer.py · 67fabc55
  zihanl authored Dec 01, 2021
  
  67fabc55
26 Nov, 2021 1 commit
- fix training.py · b1a6d73b
  zihanl authored Nov 25, 2021
  
  b1a6d73b
24 Nov, 2021 1 commit
- Add support for signal-based dynamic checkpointing · dbe6c728
  Szymon Migacz authored Nov 23, 2021
  
  dbe6c728
23 Nov, 2021 1 commit
- removed assert that prevented virtual pipelinining to be used with distributed checkpointing · 1b28a51b
  Lawrence McAfee authored Nov 23, 2021
  
  1b28a51b
22 Nov, 2021 4 commits
- assert minimum pytorch version for distributed checkpoint activations · be0da070
  Lawrence McAfee authored Nov 22, 2021
  
  be0da070
- removed unused 'get_args' import · 941a793f
  Lawrence McAfee authored Nov 22, 2021
  
  941a793f
- removed distribute_checkpointed_activations_helper() · 5993f04b
  Lawrence McAfee authored Nov 22, 2021
  
  5993f04b
- restore to original · 5fd4fd28
  zihanl authored Nov 21, 2021
  
  5fd4fd28