Commits · 8df49e72a969387786cd45d63cdc9f937ff8c8a0 · OpenDAS / Megatron-LM

26 Jul, 2022 1 commit
- added a flag to be able to switch between pytorch and ring exchange p2p · e36cdd79
  Mohammad Shoeybi authored Jul 26, 2022
  
  e36cdd79
24 May, 2022 1 commit
- resolved review comments · 739cb43d
  Vijay Korthikanti authored May 24, 2022
  
  739cb43d
17 May, 2022 4 commits
- address review comment · a7a12f82
  Vijay Korthikanti authored May 17, 2022
  
  a7a12f82
- address review comments · d19e8ad7
  Vijay Korthikanti authored May 16, 2022
  
  d19e8ad7
- address review comments · bbab79f8
  Vijay Korthikanti authored May 16, 2022
  
  bbab79f8
- address review comments · 356eb36a
  Vijay Korthikanti authored May 16, 2022
  
  356eb36a
28 Apr, 2022 2 commits
- avoid 'reverse checkpointed' term in argument description · 00129014
  Vijay Korthikanti authored Apr 28, 2022
  
  00129014
- address review comments · cfd2e216
  Vijay Korthikanti authored Apr 28, 2022
  
  cfd2e216
23 Mar, 2022 1 commit
- removed mem savings factor. +cleanup. · 11581195
  Lawrence McAfee authored Mar 23, 2022
  
  11581195
18 Mar, 2022 3 commits
- cleanup. · 3d433e8a
  Lawrence McAfee authored Mar 18, 2022
  
  3d433e8a
- mem savings factor training; need to check loss. · ed90a1b2
  Lawrence McAfee authored Mar 18, 2022
  
  ed90a1b2
- added arg '--distrib-opt-comm-mem-savings'. · 06b9ebe0
  Lawrence McAfee authored Mar 18, 2022
  
  06b9ebe0
14 Mar, 2022 1 commit
- cleaned arguments.py. · e30ad67e
  Lawrence McAfee authored Mar 14, 2022
  
  e30ad67e
11 Mar, 2022 1 commit
- checkpoint support for core attention · 53718d4c
  Vijay Korthikanti authored Mar 11, 2022
  
  53718d4c
07 Mar, 2022 1 commit
- fixes to main merge · 269f28f7
  Vijay Korthikanti authored Mar 07, 2022
  
  269f28f7
19 Feb, 2022 2 commits
- tensor model parallelism memory optmization · 5d4689c4
  Vijay Korthikanti authored Feb 18, 2022
  
  5d4689c4
- Making loading arguments from checkpoint cleaner and available more broadly. · 942c402d
  Jared Casper authored Feb 18, 2022
  
  942c402d
18 Feb, 2022 1 commit
- support fp32 training and fix embedding update · b5726555
  Sangkug Lym authored Feb 18, 2022
  
  b5726555
16 Feb, 2022 1 commit

Sangkug Lym authored Feb 13, 2022

remove redundant linear layer class definition

add fuse_gradient_accumulation attribute to weights for simple targetting

reflect feedback and clean up the codes

arg change

83b1e42f

15 Feb, 2022 4 commits
- adress review comments · 488f8c02
  Vijay Korthikanti authored Feb 11, 2022
  
  488f8c02
- minor fixes · 6f3bf9c0
  Vijay Korthikanti authored Feb 01, 2022
  
  6f3bf9c0
- vision third phase merge: pretraining methods + mit,swin backbones · 48c2a144
  Vijay Korthikanti authored Jan 31, 2022
  
  48c2a144
- wip; switching to grad-buffer-centric design · cb6f96b6
  Lawrence McAfee authored Feb 15, 2022
  
  cb6f96b6
11 Feb, 2022 1 commit
- adress review comments · 4554c3fe
  Vijay Korthikanti authored Feb 11, 2022
  
  4554c3fe
10 Feb, 2022 1 commit
- working on Float16DistributedOptimizer · 329fe582
  Lawrence McAfee authored Feb 10, 2022
  
  329fe582
04 Feb, 2022 1 commit
- renamed argument; 'embed' -> 'embedding' · c04c4977
  Lawrence McAfee authored Feb 04, 2022
  
  c04c4977
01 Feb, 2022 3 commits
- comments, cleanup. · b93bef00
  Lawrence McAfee authored Feb 01, 2022
  
  b93bef00
- minor fixes · e1f9c3a5
  Vijay Korthikanti authored Feb 01, 2022
  
  e1f9c3a5
- vision third phase merge: pretraining methods + mit,swin backbones · 01a82723
  Vijay Korthikanti authored Jan 31, 2022
  
  01a82723
28 Jan, 2022 1 commit
- changing class name AnnealingLR to OptimizerParamScheduler · 04ecc834
  Vijay Korthikanti authored Jan 28, 2022
  
  04ecc834
27 Jan, 2022 1 commit
- address review comments · 53931b8b
  Vijay Korthikanti authored Jan 27, 2022
  
  53931b8b
26 Jan, 2022 1 commit
- address review comments · 8acbbe25
  Vijay Korthikanti authored Jan 26, 2022
  
  8acbbe25
25 Jan, 2022 1 commit
- working with interleaving · 804ed2e6
  Lawrence McAfee authored Jan 24, 2022
  
  804ed2e6
24 Jan, 2022 3 commits
- added args.transformer_pipeline_model_parallel_size · a06af061
  Lawrence McAfee authored Jan 24, 2022
  
  a06af061
- fixed args.virtual_pipeline_model_parallel_size · c2b7d0b3
  Lawrence McAfee authored Jan 24, 2022
  
  c2b7d0b3
- working when no interleaving · 33dc8e9c
  Lawrence McAfee authored Jan 24, 2022
  
  33dc8e9c
22 Jan, 2022 1 commit
- second phase of vision code merge · d8c85650
  Vijay Korthikanti authored Jan 21, 2022
  
  d8c85650
19 Jan, 2022 1 commit
- flag for data parallel random initialization · a7ee77ea
  Vijay Korthikanti authored Jan 19, 2022
  
  a7ee77ea
13 Jan, 2022 1 commit
- removed argument flag; also, converted torch.FloatTenstor -> torch.empty in previous commit · 18846a0a
  Lawrence McAfee authored Jan 13, 2022
  
  18846a0a
12 Jan, 2022 1 commit
- Phase1 merge: vit optimizations + dataset enhancements + scaled_softmax kernel · 7a77abd9
  Vijay Korthikanti authored Jan 12, 2022
  
  7a77abd9