Commits · 06fc51cef50fded88e0142f32b40dc615e39672a · OpenDAS / Megatron-LM

12 Jan, 2022 1 commit
- Phase1 merge: vit optimizations + dataset enhancements + scaled_softmax kernel · 7a77abd9
  Vijay Korthikanti authored Jan 12, 2022
  
  7a77abd9
12 Oct, 2021 1 commit
- Update layer_norm_cuda_kernel.cu · f1295380
  Eddie Yan authored Oct 12, 2021
  
  f1295380
07 Oct, 2021 1 commit
- Update layer_norm_cuda_kernel.cu · 4831071c
  Eddie Yan authored Oct 06, 2021
  
  4831071c
05 Oct, 2021 1 commit
- use newer DeviceUtils header · 75e521a0
  Masaki Kozuki authored Oct 06, 2021
  
  75e521a0
23 Aug, 2021 4 commits
- removed unused variable · bc7b3539
  mshoeybi authored Aug 22, 2021
  
  bc7b3539
- chagne PR by reviews · c6b77639
  hyunwoongko authored Aug 18, 2021
  
  c6b77639
- Remove useless include · 116820a5
  hyunwoongko authored Aug 13, 2021
  
  116820a5
- Improve and fix bugs about fused softmax layer · 40becfc9
  hyunwoongko authored Aug 13, 2021
  
  40becfc9
12 Aug, 2021 2 commits
- simd fix for upper traingular softmax kernel · 62424a74
  Vijay Korthikanti authored Aug 12, 2021
  
  62424a74
- fused kernels simd fix for short sequences · 5e22689a
  Vijay Korthikanti authored Aug 12, 2021
  
  5e22689a
19 Mar, 2021 1 commit
- Bfloat fused softmax + fused layer norm · 0fa7175f
  Mohammad Shoeybi authored Mar 19, 2021
  
  0fa7175f
18 Mar, 2021 1 commit
- refactored the fused kernels build · 0d5188c1
  Mohammad Shoeybi authored Mar 17, 2021
  
  0d5188c1
17 Mar, 2021 1 commit
- fixes to upper triangular masked softmax fusion kernel · 3b12ab15
  Vijay Korthikanti authored Mar 16, 2021
  
  3b12ab15
16 Mar, 2021 2 commits
- minor fixes · 531152d9
  Vijay Korthikanti authored Mar 16, 2021
  
  531152d9
- softmax data load/store optimization · b1a83375
  Vijay Korthikanti authored Mar 16, 2021
  
  b1a83375
09 Jan, 2021 1 commit
- decoder support in transformers · 4b506832
  Vijay Korthikanti authored Jan 08, 2021
  
  4b506832
22 Dec, 2020 1 commit
- Add residual connection in fp32 machinery · 8bed1d63
  mohammad authored Dec 21, 2020
  
  8bed1d63
19 Dec, 2020 1 commit
- Rename --batch-size to --micro-batch-size and drop in-minibatch from... · 9019bbf4
  mohammad authored Dec 06, 2020
```
Rename --batch-size to --micro-batch-size and drop in-minibatch from --num-micro-batches-in-minibatch
```
  9019bbf4
13 Oct, 2020 1 commit
- suppress kernel loading output · 8fca9b49
  Vijay Korthikanti authored Oct 12, 2020
  
  8fca9b49
08 Oct, 2020 1 commit
- Create build directory for fused_kernels before building. · 54ded172
  Jared Casper authored Oct 07, 2020
  
  54ded172
06 Oct, 2020 1 commit
- build directory · 3dcd7dc0
  Vijay Korthikanti authored Oct 05, 2020
  
  3dcd7dc0
05 Oct, 2020 2 commits
- build_directory argument · a7f4e460
  Vijay Korthikanti authored Oct 05, 2020
  
  a7f4e460
- avoid non deterministic arch order in compilation · 37d77ca0
  Vijay Korthikanti authored Oct 05, 2020
  
  37d77ca0
18 Sep, 2020 1 commit
- scaling-generalized masking-softmax fusion kernel · d73def35
  root authored Sep 18, 2020
  
  d73def35
11 Sep, 2020 1 commit
- Various speed optimizations. · 51a2e6b0
  Vijay Korthikanti authored Sep 11, 2020
  
  51a2e6b0