Commits · b6b7ba4d7c064e8a8e2227efdf40d75ca8334e3d · OpenDAS / Megatron-LM

27 Aug, 2021 1 commit
- Added generate_samples_eval function · b6b7ba4d
  rprenger authored Aug 27, 2021
  
  b6b7ba4d
26 Aug, 2021 3 commits
- Fixing bug caused by merge · 448cb299
  rprenger authored Aug 25, 2021
  
  448cb299
- Found a bug. If you don't make this change and you ask for 1 token you get 2 etc. · 8694c7b0
  rprenger authored Aug 25, 2021
  
  8694c7b0
- Addressing comments · 055a673e
  rprenger authored Aug 25, 2021
  
  055a673e
23 Aug, 2021 6 commits
- removed unused variable · bc7b3539
  mshoeybi authored Aug 22, 2021
  
  bc7b3539
- Modify comments · b0b017ff
  hyunwoongko authored Aug 18, 2021
  
  b0b017ff
- chagne PR by reviews · c6b77639
  hyunwoongko authored Aug 18, 2021
  
  c6b77639
- fix bugs in fused softmax · 0d350c8d
  hyunwoongko authored Aug 13, 2021
  
  0d350c8d
- Remove useless include · 116820a5
  hyunwoongko authored Aug 13, 2021
  
  116820a5
- Improve and fix bugs about fused softmax layer · 40becfc9
  hyunwoongko authored Aug 13, 2021
  
  40becfc9
19 Aug, 2021 4 commits
- update readme and arguement definition · 99f47676
  slym authored Aug 19, 2021
  
  99f47676
- Checkpoint a set number of invidividual Transformer layers · c1e0689d
  slym authored Aug 10, 2021
```
consider the case of pipeline-model prallelism

clean up arugments

argument naming cleanup

update readme and examples
```
  c1e0689d
- pushed a fix for torch ddp · 6a0ef5b1
  mshoeybi authored Aug 18, 2021
  
  6a0ef5b1
- made contiguous buffer in local ddp default · e8fb052f
  mshoeybi authored Aug 18, 2021
  
  e8fb052f
18 Aug, 2021 2 commits
- set grads to none for the contig. buffers · df6e3cd7
  mshoeybi authored Aug 18, 2021
  
  df6e3cd7
- fixed flag name bug · db08a636
  Lawrence McAfee authored Aug 18, 2021
  
  db08a636
17 Aug, 2021 4 commits
- minor changes from github issues · 60078cf3
  mshoeybi authored Aug 17, 2021
  
  60078cf3
- updated argument name · 52b2296b
  Lawrence McAfee authored Aug 17, 2021
  
  52b2296b
- simplified the iteration read check across ranks · ede0a58f
  mshoeybi authored Aug 17, 2021
  
  ede0a58f
- added across rank sync for checkpoint iteration laod, fixed type for timing,... · 0722f878
  mshoeybi authored Aug 16, 2021
```
added across rank sync for checkpoint iteration laod, fixed type for timing, and validation iterations
```
  0722f878
16 Aug, 2021 4 commits
- changed torch distributed init method from tcp to env · ee4fd509
  Mohammad Shoeybi authored Aug 16, 2021
  
  ee4fd509
- Destroy more groups in `destroy_model_parallel` · eddf7593
  eqy authored Aug 16, 2021
```
Some tests expect a clean model parallel slate and complain if a previous test left something behind; this change clears more variables that the tests complain about.
```
  eddf7593
- added evaluation logic; finalized flag levels · 9dec5374
  Lawrence McAfee authored Aug 16, 2021
  
  9dec5374
- added flag/logic for emptying unused memory · 3bd2e973
  Lawrence McAfee authored Aug 16, 2021
  
  3bd2e973
12 Aug, 2021 2 commits
- simd fix for upper traingular softmax kernel · 62424a74
  Vijay Korthikanti authored Aug 12, 2021
  
  62424a74
- fused kernels simd fix for short sequences · 5e22689a
  Vijay Korthikanti authored Aug 12, 2021
  
  5e22689a
11 Aug, 2021 1 commit
- added asserts/checks for local ddp and params_have_main_grad · f597f02e
  Lawrence McAfee authored Aug 11, 2021
  
  f597f02e
09 Aug, 2021 1 commit
- removed saving of input/output tensors for forward-only passes of pipeline schedules · 0865c4dc
  Lawrence McAfee authored Aug 09, 2021
  
  0865c4dc
29 Jul, 2021 1 commit
- Use timers kwargs correctly to prevent bug with new p2p_communication API · e727de99
  Deepak Narayanan authored Jul 29, 2021
  
  e727de99
27 Jul, 2021 1 commit
- Make it possible to pass in tensor shapes to communication methods in p2p_communication.py · 1dccefd8
  Mostofa Patwary authored Jul 27, 2021
  
  1dccefd8
21 Jul, 2021 1 commit
- Use helper method in megatron/schedules.py as intended · 77bff386
  Deepak Narayanan authored Jul 21, 2021
  
  77bff386
20 Jul, 2021 2 commits
- Made some simplifications that keep it from hanging · 59414b33
  rprenger authored Jul 20, 2021
  
  59414b33
- fix typo in mappings.py · 92636ee1
  Taebum Kim authored Jul 20, 2021
  
  92636ee1
19 Jul, 2021 1 commit
- Removing bug possibilities and adding timing info · 7a9c4a03
  rprenger authored Jul 19, 2021
  
  7a9c4a03
16 Jul, 2021 1 commit
- (conditionally) release grad/main_grad memory in copy_model_grads_to_main_grads, after copy · 6f2bff5c
  Lawrence McAfee authored Jul 16, 2021
  
  6f2bff5c
13 Jul, 2021 1 commit
- fixed help message; removed redundant destination variable · bc5a8e20
  Lawrence McAfee authored Jul 13, 2021
  
  bc5a8e20
12 Jul, 2021 3 commits
- added comment explaining why fp32_from_float16_groups should be zeroed here · 4e64903d
  Lawrence McAfee authored Jul 12, 2021
  
  4e64903d
- switched tensorboard memory logging from opt-out to opt-in · 236a5ec0
  Lawrence McAfee authored Jul 12, 2021
  
  236a5ec0
- added memory stats (allocated/reserved) to tensorboard logging · 24b7c3c0
  Lawrence McAfee authored Jul 12, 2021
  
  24b7c3c0
09 Jul, 2021 1 commit
- Refactoring code so server code is more independent of sampling and adding a... · 29dd0a35
  rprenger authored Jul 09, 2021
```
Refactoring code so server code is more independent of sampling and adding a CLI.  CLI still has URL of server hard-coded
```
  29dd0a35