Commits · bbab79f86907a877e8041c757c8ed8e8312a15df · OpenDAS / Megatron-LM

17 May, 2022 2 commits
- address review comments · bbab79f8
  Vijay Korthikanti authored May 16, 2022
  
  bbab79f8
- address review comments · 356eb36a
  Vijay Korthikanti authored May 16, 2022
  
  356eb36a
16 May, 2022 1 commit
- cleanup imports in transformer.py · 28b6ffce
  Vijay Korthikanti authored May 16, 2022
  
  28b6ffce
09 May, 2022 1 commit
- (1)support pyt20.12 compatibility, (2) arg naming update · f602ac56
  Sangkug Lym authored Apr 11, 2022
```
change dummy_handler to nullcontext
```
  f602ac56
28 Apr, 2022 1 commit
- address review comments · cfd2e216
  Vijay Korthikanti authored Apr 28, 2022
  
  cfd2e216
30 Mar, 2022 1 commit
- sequence parallelism for embedding dropout and last linear layer + memory optimizations · eec218d8
  Vijay Korthikanti authored Mar 30, 2022
  
  eec218d8
25 Mar, 2022 1 commit
- jit functions warmpups to (1) match fprop and recompute results and (2) remove grad_enable · c7fef593
  Sangkug Lym authored Mar 24, 2022
  
  c7fef593
15 Mar, 2022 2 commits
- fixed. · 65e6bc32
  Lawrence McAfee authored Mar 15, 2022
  
  65e6bc32
- debugging. · 4c598f9d
  Lawrence McAfee authored Mar 15, 2022
  
  4c598f9d
11 Mar, 2022 1 commit
- checkpoint support for core attention · 53718d4c
  Vijay Korthikanti authored Mar 11, 2022
  
  53718d4c
09 Mar, 2022 1 commit
- bug fixes · dd96d402
  Vijay Korthikanti authored Mar 08, 2022
  
  dd96d402
07 Mar, 2022 2 commits
- fixes to main merge · 269f28f7
  Vijay Korthikanti authored Mar 07, 2022
  
  269f28f7
- refactor to help merge with main · 0d77c0e9
  Vijay Korthikanti authored Mar 07, 2022
  
  0d77c0e9
03 Mar, 2022 1 commit
- get sequence parallelism to work with pipeline parallelism · 6658158b
  Vijay Korthikanti authored Mar 02, 2022
  
  6658158b
02 Mar, 2022 1 commit
- layernorm grad sync + name chnages · c0f10643
  Vijay Korthikanti authored Mar 02, 2022
  
  c0f10643
01 Mar, 2022 1 commit
- Fourth phase vision merge: classification and segmentation tasks · ca7a2e34
  Vijay Korthikanti authored Feb 28, 2022
  
  ca7a2e34
19 Feb, 2022 1 commit
- tensor model parallelism memory optmization · 5d4689c4
  Vijay Korthikanti authored Feb 18, 2022
  
  5d4689c4
18 Feb, 2022 1 commit
- support fp32 training and fix embedding update · b5726555
  Sangkug Lym authored Feb 18, 2022
  
  b5726555
16 Feb, 2022 1 commit

Sangkug Lym authored Feb 13, 2022

remove redundant linear layer class definition

add fuse_gradient_accumulation attribute to weights for simple targetting

reflect feedback and clean up the codes

arg change

83b1e42f

15 Feb, 2022 4 commits
- adress review comments · 488f8c02
  Vijay Korthikanti authored Feb 11, 2022
  
  488f8c02
- more minor fixes · 1cd3650d
  Vijay Korthikanti authored Feb 01, 2022
  
  1cd3650d
- minor fixes · 6f3bf9c0
  Vijay Korthikanti authored Feb 01, 2022
  
  6f3bf9c0
- vision third phase merge: pretraining methods + mit,swin backbones · 48c2a144
  Vijay Korthikanti authored Jan 31, 2022
  
  48c2a144
11 Feb, 2022 1 commit
- adress review comments · 4554c3fe
  Vijay Korthikanti authored Feb 11, 2022
  
  4554c3fe
04 Feb, 2022 1 commit
- renamed argument; 'embed' -> 'embedding' · c04c4977
  Lawrence McAfee authored Feb 04, 2022
  
  c04c4977
03 Feb, 2022 1 commit
- Removing unnecessary permutes and scatter/gather · a0bea425
  rprenger authored Feb 03, 2022
  
  a0bea425
01 Feb, 2022 5 commits
- comments, cleanup. · b93bef00
  Lawrence McAfee authored Feb 01, 2022
  
  b93bef00
- more minor fixes · 3f1a728a
  Vijay Korthikanti authored Feb 01, 2022
  
  3f1a728a
- found root source of t5 issue (fast layer norm) · bea16fa3
  Lawrence McAfee authored Feb 01, 2022
  
  bea16fa3
- minor fixes · e1f9c3a5
  Vijay Korthikanti authored Feb 01, 2022
  
  e1f9c3a5
- vision third phase merge: pretraining methods + mit,swin backbones · 01a82723
  Vijay Korthikanti authored Jan 31, 2022
  
  01a82723
31 Jan, 2022 1 commit
- added 'no-op' layer, to replace transformer layer when num_layers == 0. · 1fa6990c
  Lawrence McAfee authored Jan 31, 2022
  
  1fa6990c
29 Jan, 2022 1 commit
- narrowed issue to pipeline rank 0, virtual pipeline rank >= 1 · 5bc9f889
  Lawrence McAfee authored Jan 28, 2022
  
  5bc9f889
27 Jan, 2022 1 commit
- address review comments · 53931b8b
  Vijay Korthikanti authored Jan 27, 2022
  
  53931b8b
26 Jan, 2022 1 commit
- further clarified viewless tensor comment in transformer.py · d16e2a24
  Lawrence McAfee authored Jan 26, 2022
  
  d16e2a24
25 Jan, 2022 1 commit
- limit 'make_viewless_tensor()' to case of micro_batch_size == 1; added comment · 24369dd6
  Lawrence McAfee authored Jan 25, 2022
  
  24369dd6
24 Jan, 2022 2 commits
- fixed args.virtual_pipeline_model_parallel_size · c2b7d0b3
  Lawrence McAfee authored Jan 24, 2022
  
  c2b7d0b3
- minor fixes · 7e810e41
  Vijay Korthikanti authored Jan 24, 2022
  
  7e810e41
22 Jan, 2022 1 commit
- second phase of vision code merge · d8c85650
  Vijay Korthikanti authored Jan 21, 2022
  
  d8c85650
13 Jan, 2022 1 commit
- minor fixes · 3b29f0e7
  Vijay Korthikanti authored Jan 13, 2022
  
  3b29f0e7