- 28 Apr, 2023 1 commit
-
-
Kirthi Shankar Sivamani authored
Signed-off-by:Kirthi Shankar Sivamani <ksivamani@nvidia.com>
-
- 26 Apr, 2023 1 commit
-
-
Przemyslaw Tredak authored
* Faster split of QKV for FlashAttention Signed-off-by:
Przemek Tredak <ptredak@nvidia.com> * CI fixes Signed-off-by:
-
- 22 Apr, 2023 1 commit
-
-
asfiyab-nvidia authored
add cast for BF16 input/output types for Q/DQ ONNX ops Signed-off-by:
Asfiya Baig <asfiyab@nvidia.com> Co-authored-by:
-
- 21 Apr, 2023 4 commits
-
-
Kirthi Shankar Sivamani authored
Signed-off-by: -
Sangkug Lym authored
Signed-off-by:
Sangkug Lym <slym@nvidia.com> Co-authored-by:
-
cyanguwa authored
* Add FP8 fused attention to TE for PyTorch Signed-off-by:
Charlene Yang <charleney@nvidia.com> * add license for cudnn-frontend, modify installation requirements, and refactor some headers for aesthetics Signed-off-by:
-
Kirthi Shankar Sivamani authored
* Initial refactor; linker error Signed-off-by:
Tim Moon <tmoon@nvidia.com> * Lint Signed-off-by:
-
- 19 Apr, 2023 1 commit
-
-
Kirthi Shankar Sivamani authored
* Port initial changes Co-authored-by:
Vasudevan Rengasamy <vrengasamy@nvidia.com> Signed-off-by:
-
- 18 Apr, 2023 1 commit
-
-
Sangkug Lym authored
* amax reduction internval Signed-off-by:
-
- 17 Apr, 2023 1 commit
-
-
Kirthi Shankar Sivamani authored
* use upstream flash-attn Signed-off-by:
-
- 14 Apr, 2023 1 commit
-
-
Kirthi Shankar Sivamani authored
Signed-off-by:
-
- 13 Apr, 2023 1 commit
-
-
Neta Zmora authored
* Fix model load exception when state resides on GPU - Whenever converting a torch.tensor to numpy, we need to first migrate the tensor storage to the host CPU. - Add a warning not to do contant-folding when exporting to ONNX. This is due to a torch.onnx export bug. - Refactor compare_outputs Signed-off-by:
Neta Zmora <nzmora@nvidia.com> * Onnx export: Improve remark text Signed-off-by:
-
- 07 Apr, 2023 2 commits
-
-
ngoyal2707 authored
* made bias configurable Signed-off-by:
Naman Goyal <naman@fb.com> * removed commented lines Signed-off-by:
Tim Moon <4406448+timmoon10@users.noreply.github.com> Signed-off-by:
ngoyal2707 <ngoyal2707@users.noreply.github.com> * Update transformer_engine/pytorch/jit.py Co-authored-by:
-
Kirthi Shankar Sivamani authored
* small cleanup before starting Signed-off-by:
-
- 05 Apr, 2023 1 commit
-
-
Sangkug Lym authored
* async amax reduction add env knob to enable async amax reduction Signed-off-by:
slym <slym@login-preos01.a51.clusters.nvidia.com> * Style fixes Signed-off-by:
-
- 04 Apr, 2023 1 commit
-
-
Kirthi Shankar Sivamani authored
* Add FP8 support for Ada Signed-off-by:
-
- 03 Apr, 2023 1 commit
-
-
galagam authored
* Bugfix - compute scale_inv when loading checkpoint Signed-off-by:
Gal Hubara Agam <ghubaraagam@nvidia.com> * Save inverse scale in extra state tensor + minor CR fixes Signed-off-by:
-
- 30 Mar, 2023 2 commits
-
-
Kirthi Shankar Sivamani authored
* Fix segfault during GeLU export Signed-off-by:
-
Kirthi Shankar Sivamani authored
* Change FP8 recipe defaults Signed-off-by:
-
- 29 Mar, 2023 1 commit
-
-
tcherckez-nvidia authored
Signed-off-by:
Tal Cherckez <tcherckez@nvidia.com> Signed-off-by:
-
- 28 Mar, 2023 3 commits
-
-
vasunvidia authored
* Add support for fp8 GEMM BIAS AUX GELU fusion Signed-off-by:
-
Tim Moon authored
* Remove zombie process from querying TE install path Co-authored-by:
-
Kirthi Shankar Sivamani authored
* fix usage of return_bias argument Signed-off-by:
-
- 23 Mar, 2023 1 commit
-
-
Neta Zmora authored
* Fix GELU ONNX export * Wrap GELU export with cast to/from FP32 to achieve same compute precision as TE. * Increase GELU export test thresholds. * Change export to ONNX opset 17 for smaller representation of LN (single node instead of subgraph). * Remove the need for LN work-around for ORT Signed-off-by:
-
- 22 Mar, 2023 1 commit
-
-
Kirthi Shankar Sivamani authored
FA doesn't support compute 8.6 with head_dim>64 Signed-off-by:
-
- 21 Mar, 2023 1 commit
-
-
vasunvidia authored
* Initial commit for fp8_transpose_dbias kernel Signed-off-by:
-
- 16 Mar, 2023 2 commits
-
-
Kirthi Shankar Sivamani authored
Signed-off-by: -
Neta Zmora authored
* Add a temporary workaround to layernorm export Seems like ORT is performing template-matching for LN and incorrectly concludes that it doesn't have a kernel for FP32 LN. The work-around adds the addition of fake_zero which is meant to prevent the template matching while keeping the graph virtually unchanged. This also requires `do_constant_folding=False` in `torch.onnx.export`. Signed-off-by:
-
- 15 Mar, 2023 1 commit
-
-
Kirthi Shankar Sivamani authored
Use updated comm API PyTorch Signed-off-by:
-
- 14 Mar, 2023 2 commits
-
-
Ming-Xu Huang authored
* Updated TE/JAX docs Signed-off-by:
Ming Huang <mingh@nvidia.com> * Adding TE/JAX docs' rst files Signed-off-by:
-
Kirthi Shankar Sivamani authored
* Catch FP8 modulo16 error before cublas and fp8 kernels Signed-off-by:
-
- 13 Mar, 2023 1 commit
-
-
Kirthi Shankar Sivamani authored
* catch incorrect usage of fp8_autocast Signed-off-by:
-
- 11 Mar, 2023 2 commits
-
-
Przemyslaw Tredak authored
* Change from AutoDoc to AutoAPI Signed-off-by:
-
Kirthi Shankar Sivamani authored
* deprecate qk layer scaling and fp32 softmax args Signed-off-by:
-
- 07 Mar, 2023 1 commit
-
-
Kirthi Shankar Sivamani authored
* ignore self attention mask for causal type Signed-off-by:
-
- 02 Mar, 2023 1 commit
-
-
Kirthi Shankar Sivamani authored
* fix qkv weight unfused path Signed-off-by:
-
- 25 Feb, 2023 1 commit
-
-
Kirthi Shankar Sivamani authored
Signed-off-by:
-
- 24 Feb, 2023 1 commit
-
-
Kirthi Shankar Sivamani authored
* Remove redundant amax AR for SP case Signed-off-by:
-
- 23 Feb, 2023 1 commit
-
-
Tim Moon authored
* Deprecate fp32_output option for PyT linear layers Automatically detect dtype for user-provided output tensors. Signed-off-by:
-
- 22 Feb, 2023 1 commit
-
-
cyanguwa authored
* add flash attention to TransformerLayer Signed-off-by:
cyanguwa <cyang.uwa@gmail.com> Signed-off-by:
-
