Commits · edf475ff08ad9fdc97167969a6a857dd1cc12cdc · OpenDAS / Megatron-LM

23 Feb, 2021 4 commits
- added script for creating embeddings · edf475ff
  Mostofa Patwary authored Feb 23, 2021
  
  edf475ff
- Merge branch 'main' into main_retriver_merge_ict_eval · 60c95ab6
  Mostofa Patwary authored Feb 23, 2021
  
  60c95ab6
- Added code for building embeddings and savings · bcd605f8
  Mostofa Patwary authored Feb 23, 2021
  
  bcd605f8
- Merge branch 'main_dedup' into 'main' · a1f4032b
  Jared Casper authored Feb 22, 2021
```
Storing and loading fingerprints of in deduplication

See merge request ADLR/megatron-lm!236
```
  a1f4032b
22 Feb, 2021 3 commits
- addressed the comments · ebc4a408
  Mostofa Patwary authored Feb 22, 2021
  
  ebc4a408
- Merge branch 'interleaved_schedule_assertion_fix' into 'main' · 90623142
  Jared Casper authored Feb 22, 2021
```
Fix interleaved schedule assertion

See merge request ADLR/megatron-lm!237
```
  90623142
- Fix interleaved schedule assertion · b7067cbd
  Deepak Narayanan authored Feb 22, 2021
  
  b7067cbd
19 Feb, 2021 5 commits
- Merge branch 'main_retriver_merge' into main_retriver_merge_ict_eval · 31d39ec0
  Mostofa Patwary authored Feb 19, 2021
  
  31d39ec0
- updated the readme · 5a867963
  Mostofa Patwary authored Feb 19, 2021
  
  5a867963
- now multiple pickle files can be added · 4b821fb5
  Mostofa Patwary authored Feb 19, 2021
  
  4b821fb5
- Added an example · f6ddeca0
  Mostofa Patwary authored Feb 18, 2021
  
  f6ddeca0
- added storing and loading of pickle file of hash · 2f6e46bd
  Mostofa Patwary authored Feb 18, 2021
  
  2f6e46bd
18 Feb, 2021 10 commits
- Merge branch 'main_jaccard' into 'main' · 9dc111cc
  Jared Casper authored Feb 18, 2021
```
Bug Jaccard similarity and filtering n-grams

See merge request ADLR/megatron-lm!234
```
  9dc111cc
- Merge branch 'main_retriver_merge' into 'main' · dfd428cb
  Jared Casper authored Feb 18, 2021
```
ICT Retriever

See merge request ADLR/megatron-lm!235
```
  dfd428cb
- addressed the comments · 3f3ba5e7
  Mostofa Patwary authored Feb 18, 2021
  
  3f3ba5e7
- added readme · a449d312
  Mostofa Patwary authored Feb 18, 2021
  
  a449d312
- fixed an issue with ICT · 3d96729a
  Mostofa Patwary authored Feb 17, 2021
  
  3d96729a
- Merge branch 'main_retriver_merge' into 'main' · 68233932
  Jared Casper authored Feb 17, 2021
```
Merge ICT Retriever with Main branch

See merge request ADLR/megatron-lm!227
```
  68233932
- fixed a minor bug · 816fb890
  Mostofa Patwary authored Feb 17, 2021
  
  816fb890
- pulled latest megatron · b2f57fc4
  Mostofa Patwary authored Feb 17, 2021
  
  b2f57fc4
- evaluation code ongoing · 612f438a
  Mostofa Patwary authored Feb 17, 2021
  
  612f438a
- Merge branch 'better_schedule' into 'main' · 76e3fca0
  Jared Casper authored Feb 17, 2021
```
Interleaved pipelined execution and refactored schedule / communication code

See merge request ADLR/megatron-lm!189
```
  76e3fca0
17 Feb, 2021 4 commits
- Merge branch 'main_retriver_merge' into main_retriver_merge_ict_eval · ebc95c35
  Mostofa Patwary authored Feb 17, 2021
  
  ebc95c35
- filtering n-grams · 59372322
  Mostofa Patwary authored Feb 17, 2021
  
  59372322
- Fixed a minor bug · 0a158128
  Mostofa Patwary authored Feb 17, 2021
  
  0a158128
- Turn scatter-gather optimization on by default · 72e4230b
  Deepak Narayanan authored Feb 16, 2021
  
  72e4230b
15 Feb, 2021 3 commits
- updated script as well · a4b628ab
  Mostofa Patwary authored Feb 15, 2021
  
  a4b628ab
- addressed the comments given by Mohammad · 447c1171
  Mostofa Patwary authored Feb 15, 2021
  
  447c1171
- minor change · 360fb6b7
  Mostofa Patwary authored Feb 15, 2021
  
  360fb6b7
13 Feb, 2021 2 commits
- Change argument to control the number of model chunks in a stage · dcef9069
  Deepak Narayanan authored Feb 13, 2021
  
  dcef9069
- More comments and some cleanup (e.g., better variable names) · 5489bda9
  Deepak Narayanan authored Feb 13, 2021
  
  5489bda9
11 Feb, 2021 1 commit
- Merge branch 'main_retriver_merge' into main_retriver_merge_ict_eval · 4bea4e76
  Mostofa Patwary authored Feb 11, 2021
  
  4bea4e76
10 Feb, 2021 4 commits
- Comments in megatron/schedules.py and address a few more comments · 626645c0
  Deepak Narayanan authored Feb 10, 2021
  
  626645c0
- removed the function compile_helper · 22a3d81a
  Mostofa Patwary authored Feb 09, 2021
  
  22a3d81a
- started evaluation of ICT · 613d0fe8
  Mostofa Patwary authored Feb 09, 2021
  
  613d0fe8
- Move unwrap to megatron/utils.py and clean up imports in megatron/schedules.py · cc691cbf
  Deepak Narayanan authored Feb 10, 2021
  
  cc691cbf
09 Feb, 2021 4 commits
- Compute tensor chunk size more cleanly, and add assertion for global batch size · e3e5ea89
  Deepak Narayanan authored Jan 20, 2021
  
  e3e5ea89
- Break up tensors sent between pipeline stages into smaller chunks that can be all-gathered · 27fc4689
  Deepak Narayanan authored Jan 20, 2021
  
  27fc4689
- Put in barriers in appropriate places to measure length of pipeline stall · 8e922d5b
  Deepak Narayanan authored Jan 09, 2021
  
  8e922d5b
- Interleaved pipeline execution and code refactoring · dd889062
  Deepak Narayanan authored Dec 12, 2020
```
- Split a model's computation into multiple virtual stages as needed,
and schedule communication correctly between these virtual stages
- Move schedule code into `schedules.py` and communication code into
`p2p_communication.py`
- Use hyphens instead of spaces in all time logging for consistency
- Factor out code in megatron/training.py into helper functions
- Refactor evaluate() function: make it use forward_backward_schedule
functions
```
  dd889062