Commits · 9ad1944db1f97000377dc5aee36dcd656b1ae4a2 · OpenDAS / Megatron-LM

26 May, 2022 1 commit
- bug fix for get_data_parallel_src_rank · 9ad1944d
  Vijay Korthikanti authored May 26, 2022
  
  9ad1944d
24 May, 2022 3 commits
- resolved review comments · 739cb43d
  Vijay Korthikanti authored May 24, 2022
  
  739cb43d
- preallocating global buffer to avoid memory fragmentation · 9dc3c42a
  Vijay Korthikanti authored May 24, 2022
  
  9dc3c42a
- fix for sequence parallelism in bert pooling · 8474e6e5
  Vijay Korthikanti authored May 24, 2022
  
  8474e6e5
20 May, 2022 2 commits
- avoiding sequence parallelism on the pooler · 3f91f09b
  Vijay Korthikanti authored May 20, 2022
  
  3f91f09b
- bert regression fixes · 6aaafee6
  Vijay Korthikanti authored May 20, 2022
  
  6aaafee6
18 May, 2022 1 commit
- t5 regression fixes · 5d647381
  Vijay Korthikanti authored May 18, 2022
  
  5d647381
17 May, 2022 4 commits
- address review comment · a7a12f82
  Vijay Korthikanti authored May 17, 2022
  
  a7a12f82
- address review comments · d19e8ad7
  Vijay Korthikanti authored May 16, 2022
  
  d19e8ad7
- address review comments · bbab79f8
  Vijay Korthikanti authored May 16, 2022
  
  bbab79f8
- address review comments · 356eb36a
  Vijay Korthikanti authored May 16, 2022
  
  356eb36a
16 May, 2022 6 commits
- Merge branch 'main' into sequence_parallel · aae72ffc
  Vijay Korthikanti authored May 16, 2022
  
  aae72ffc
- Merge branch 'vision-merge' into 'main' · b44dca25
  Jared Casper authored May 16, 2022
```
Fourth phase vision merge: classification and segmentation tasks

See merge request ADLR/megatron-lm!400
```
  b44dca25
- address review comment · 214a9a08
  Vijay Korthikanti authored May 16, 2022
  
  214a9a08
- cleanup imports in transformer.py · 28b6ffce
  Vijay Korthikanti authored May 16, 2022
  
  28b6ffce
- update jit warmup code to handle sequence parallelism · a4ef7c40
  Vijay Korthikanti authored May 16, 2022
  
  a4ef7c40
- Merge branch 'main' into sequence_parallel · 7f9a48ba
  Vijay Korthikanti authored May 16, 2022
  
  7f9a48ba
13 May, 2022 1 commit
- address review comments · 35a3fa8b
  Vijay Korthikanti authored May 13, 2022
  
  35a3fa8b
10 May, 2022 1 commit

Merge branch 'slym/jit_warmup' into 'main' · d2394294

Jared Casper authored May 10, 2022

jit functions warmpups to (1) match fprop and recompute results and (2) remove grad_enable

See merge request ADLR/megatron-lm!404

d2394294

09 May, 2022 1 commit
- (1)support pyt20.12 compatibility, (2) arg naming update · f602ac56
  Sangkug Lym authored Apr 11, 2022
```
change dummy_handler to nullcontext
```
  f602ac56
28 Apr, 2022 3 commits
- avoid 'reverse checkpointed' term in argument description · 00129014
  Vijay Korthikanti authored Apr 28, 2022
  
  00129014
- reverting pre allocation of optimizer states; it does not seem to help with fragmentation · 5d2e13a6
  Vijay Korthikanti authored Apr 28, 2022
  
  5d2e13a6
- address review comments · cfd2e216
  Vijay Korthikanti authored Apr 28, 2022
  
  cfd2e216
25 Apr, 2022 1 commit
- make interleaving work with optimizations · 13b3dca6
  Vijay Korthikanti authored Apr 25, 2022
  
  13b3dca6
30 Mar, 2022 1 commit
- sequence parallelism for embedding dropout and last linear layer + memory optimizations · eec218d8
  Vijay Korthikanti authored Mar 30, 2022
  
  eec218d8
25 Mar, 2022 1 commit
- jit functions warmpups to (1) match fprop and recompute results and (2) remove grad_enable · c7fef593
  Sangkug Lym authored Mar 24, 2022
  
  c7fef593
15 Mar, 2022 3 commits
- Merge branch 'lmcafee/free-pipe-output-bug' into 'main' · 15f6bb1b
  Jared Casper authored Mar 15, 2022
```
Bugfix: Pipeline output deallocation w/ persistent layer norm

See merge request ADLR/megatron-lm!402
```
  15f6bb1b
- fixed. · 65e6bc32
  Lawrence McAfee authored Mar 15, 2022
  
  65e6bc32
- debugging. · 4c598f9d
  Lawrence McAfee authored Mar 15, 2022
  
  4c598f9d
11 Mar, 2022 1 commit
- checkpoint support for core attention · 53718d4c
  Vijay Korthikanti authored Mar 11, 2022
  
  53718d4c
09 Mar, 2022 1 commit
- bug fixes · dd96d402
  Vijay Korthikanti authored Mar 08, 2022
  
  dd96d402
07 Mar, 2022 3 commits
- fixes to main merge · 269f28f7
  Vijay Korthikanti authored Mar 07, 2022
  
  269f28f7
- Merge branch 'main' into allreduce-split · 6fdbf26b
  Vijay Korthikanti authored Mar 07, 2022
  
  6fdbf26b
- refactor to help merge with main · 0d77c0e9
  Vijay Korthikanti authored Mar 07, 2022
  
  0d77c0e9
03 Mar, 2022 2 commits
- column parallel linear with sequence parallelism · 02bb1f5c
  Vijay Korthikanti authored Mar 02, 2022
  
  02bb1f5c
- get sequence parallelism to work with pipeline parallelism · 6658158b
  Vijay Korthikanti authored Mar 02, 2022
  
  6658158b
02 Mar, 2022 1 commit
- layernorm grad sync + name chnages · c0f10643
  Vijay Korthikanti authored Mar 02, 2022
  
  c0f10643
01 Mar, 2022 3 commits
- more cleanup · 0c9e6a16
  Vijay Korthikanti authored Feb 28, 2022
  
  0c9e6a16
- minor cleanup · 8db0e41b
  Vijay Korthikanti authored Feb 28, 2022
  
  8db0e41b
- Fourth phase vision merge: classification and segmentation tasks · ca7a2e34
  Vijay Korthikanti authored Feb 28, 2022
  
  ca7a2e34