Commits · 45f4ee54e783fd63fa1ef0c7838bff149eaa18ae · OpenDAS / Megatron-LM

04 Aug, 2022 1 commit
- yttm + BytelevelBPE + setencepeice tokenizer support · 45f4ee54
  Vijay Korthikanti authored Aug 04, 2022
  
  45f4ee54
28 Jul, 2022 1 commit
- support for all mask in fused kernel + avoiding inplace operation in bwd pass · 76db9583
  Vijay Korthikanti authored Jul 28, 2022
  
  76db9583
26 Jul, 2022 1 commit
- added a flag to be able to switch between pytorch and ring exchange p2p · e36cdd79
  Mohammad Shoeybi authored Jul 26, 2022
  
  e36cdd79
18 Jul, 2022 1 commit
- Adding checks for total number of tokens to keep server from crashing · cfe2c2be
  rprenger authored Jul 18, 2022
  
  cfe2c2be
15 Jul, 2022 2 commits
- remove debug lines for printing · 2e65bee5
  peng xu authored Jul 15, 2022
  
  2e65bee5
- fix pipeline parallel for beam search · e5034150
  peng xu authored Jul 15, 2022
  
  e5034150
24 Jun, 2022 2 commits
- Fixing up checkpointing. · 2725dc0b
  Jared Casper authored Jun 24, 2022
  
  2725dc0b
- Remove unused function. · 4eb802c4
  Jared Casper authored Jun 24, 2022
  
  4eb802c4
21 Jun, 2022 1 commit
- Changing name of parameters from factual_decay to top_p_decay · ed6806ac
  rprenger authored Jun 21, 2022
  
  ed6806ac
08 Jun, 2022 6 commits
- Adding top_p decay and bound for factual sampling from Factuality Enhanced LMs · b49349ec
  rprenger authored Jun 08, 2022
  
  b49349ec
- fixed load_checkpoint(). · 74ba3797
  Lawrence McAfee authored Jun 08, 2022
  
  74ba3797
- error fixes & tested. · b178e6fc
  Lawrence McAfee authored Jun 08, 2022
  
  b178e6fc
- added backwards compatibility to checkpointing.py. · 977efdfb
  Lawrence McAfee authored Jun 08, 2022
  
  977efdfb
- re-added grad dtype assertion in clip_grads. · ef63acd2
  Lawrence McAfee authored Jun 08, 2022
  
  ef63acd2
- removed count-zeros debuggables. · 9044bc42
  Lawrence McAfee authored Jun 08, 2022
  
  9044bc42
01 Jun, 2022 1 commit
- testing fix w/ clip_grads. · 7fccd6a1
  Lawrence McAfee authored Jun 01, 2022
  
  7fccd6a1
26 May, 2022 1 commit
- bug fix for get_data_parallel_src_rank · 9ad1944d
  Vijay Korthikanti authored May 26, 2022
  
  9ad1944d
25 May, 2022 1 commit
- separate file for license · da11c982
  Peng Xu authored May 25, 2022
  
  da11c982
24 May, 2022 7 commits
- resolved review comments · 739cb43d
  Vijay Korthikanti authored May 24, 2022
  
  739cb43d
- allreduce layernorm fixes. · de593298
  Lawrence McAfee authored May 24, 2022
  
  de593298
- fixed for new jit warmup changes. · cf1c7848
  Lawrence McAfee authored May 24, 2022
  
  cf1c7848
- preallocating global buffer to avoid memory fragmentation · 9dc3c42a
  Vijay Korthikanti authored May 24, 2022
  
  9dc3c42a
- added timer for layernorm all-reduce. · aff99e32
  Lawrence McAfee authored May 24, 2022
  
  aff99e32
- moved SP's layernorm all-reduce into optimizer.py. · ee57e086
  Lawrence McAfee authored May 24, 2022
  
  ee57e086
- fix for sequence parallelism in bert pooling · 8474e6e5
  Vijay Korthikanti authored May 24, 2022
  
  8474e6e5
20 May, 2022 2 commits
- avoiding sequence parallelism on the pooler · 3f91f09b
  Vijay Korthikanti authored May 20, 2022
  
  3f91f09b
- bert regression fixes · 6aaafee6
  Vijay Korthikanti authored May 20, 2022
  
  6aaafee6
18 May, 2022 1 commit
- t5 regression fixes · 5d647381
  Vijay Korthikanti authored May 18, 2022
  
  5d647381
17 May, 2022 4 commits
- address review comment · a7a12f82
  Vijay Korthikanti authored May 17, 2022
  
  a7a12f82
- address review comments · d19e8ad7
  Vijay Korthikanti authored May 16, 2022
  
  d19e8ad7
- address review comments · bbab79f8
  Vijay Korthikanti authored May 16, 2022
  
  bbab79f8
- address review comments · 356eb36a
  Vijay Korthikanti authored May 16, 2022
  
  356eb36a
16 May, 2022 4 commits
- address review comment · 214a9a08
  Vijay Korthikanti authored May 16, 2022
  
  214a9a08
- cleanup imports in transformer.py · 28b6ffce
  Vijay Korthikanti authored May 16, 2022
  
  28b6ffce
- update jit warmup code to handle sequence parallelism · a4ef7c40
  Vijay Korthikanti authored May 16, 2022
  
  a4ef7c40
- Fixing beam search in distributed mode · 6be75e2a
  rprenger authored May 16, 2022
  
  6be75e2a
12 May, 2022 1 commit

Data preprocessing testing changes + fixes · a2c5e6cd

John Kamalu authored May 12, 2022

tools/merge_datasets.py
- tool to merge multiple dataset files into a single dataset
- testing conducted and included in the megatron-testing repo https://gitlab-master.nvidia.com/ADLR/megatron-testing

tools/preprocess_data.py
- magic numbers changed to required command line arguments

megatron/data/indexed_dataset.py
- when merging, fix to properly update document index
- testing conducted and included in the megatron-testing repo (see above)
- fix follows this history https://github.com/bigscience-workshop/Megatron-DeepSpeed/pull/66

a2c5e6cd

09 May, 2022 1 commit
- (1)support pyt20.12 compatibility, (2) arg naming update · f602ac56
  Sangkug Lym authored Apr 11, 2022
```
change dummy_handler to nullcontext
```
  f602ac56
05 May, 2022 1 commit
- Making the API talk to the server, fixed a bug where parameters weren't getting forwarded to node · fd176a90
  rprenger authored May 04, 2022
  
  fd176a90
04 May, 2022 1 commit
- final version · 5a570bd8
  peng xu authored May 03, 2022
  
  5a570bd8