- 15 Dec, 2023 3 commits
-
-
Przemyslaw Tredak authored
* Disable dynamo for Fused Attention Signed-off-by:
Przemek Tredak <ptredak@nvidia.com> * Added test Signed-off-by:
Tim Moon <4406448+timmoon10@users.noreply.github.com>
-
Kunlun Li authored
Avoid redeclaration error of "nv_bfloat16" when Compiling in CUDA12.1. Error log: /usr/local/cuda/include/cuda_fp16.hpp(2724): error: invalid redeclaration of type name "nv_bfloat16" (declared at line 2837 of /usr/local/cuda/include/cuda_bf16.hpp) Signed-off-by:Kunlun Li <94586211+kunlunl@users.noreply.github.com>
-
Fabian Joswig authored
[fix] fixed micro batched inference with RoPE Signed-off-by:
Fabian Joswig <fabian.joswig@deepl.com> Co-authored-by:
cyanguwa <8636796+cyanguwa@users.noreply.github.com>
-
- 14 Dec, 2023 1 commit
-
-
Alp Dener authored
applied Google-advised fix to register custom op primitives with the device dispatch list Signed-off-by:Alp Dener <adener@nvidia.com>
-
- 13 Dec, 2023 2 commits
-
-
cyanguwa authored
* fix backend selection for sm80 Signed-off-by:
-
Marks101 authored
Signed-off-by:Markus Schnoes <markus.schnoes@gmx.de>
-
- 12 Dec, 2023 2 commits
-
-
cyanguwa authored
disable pylint bare except Signed-off-by: -
cyanguwa authored
* fix onnx/dynamo error Signed-off-by:
Przemyslaw Tredak <ptrendx@gmail.com> Signed-off-by:
-
- 07 Dec, 2023 2 commits
-
-
cyanguwa authored
* Integrate cuDNN frontend v1 to fused attention and miscellaneous fixes Signed-off-by:
-
Tim Moon authored
* Float8Tensor uses cached transpose if available Signed-off-by:
Tim Moon <tmoon@nvidia.com> * Fix bug with non-2D transpose Signed-off-by:
-
- 06 Dec, 2023 2 commits
-
-
Santosh Bhavani authored
* Add H200 perf non-alpha image Signed-off-by:
Santosh Bhavani <santosh@semantic.md> * Update README.rst - non-transparent H200 plot Signed-off-by:
-
Santosh Bhavani authored
* Update README.rst - FP8 convergence - added FP8 convergence section - removed model support (to be replaced with a feature support table) Signed-off-by:
Kirthi Shankar Sivamani <ksivamani@nvidia.com> * Update README.rst Updated Latest News Signed-off-by:
-
- 04 Dec, 2023 4 commits
-
-
Zhenhuan Liu authored
Signed-off-by:Zhenhuan Liu <nkulzh16@gmail.com>
-
Przemyslaw Tredak authored
Signed-off-by: -
Marks101 authored
* [PyTorch] TransformerLayer: add parallel_attention_mlp to support Falcon models Signed-off-by:
-
zlsh80826 authored
Add checkpoint_name Signed-off-by:Reese Wang <rewang@nvidia.com>
-
- 01 Dec, 2023 3 commits
-
-
Tim Moon authored
Fix incorrect variable name in LayerNormMLP backward Signed-off-by: -
LadyRick authored
[PyTorch] fix amax calculate during fp8 calibration Signed-off-by:ladyrick <ladyrick@qq.com>
-
zlsh80826 authored
* Add rng_state output for cross fused attention Signed-off-by:
-
- 30 Nov, 2023 3 commits
-
-
Deepak Narayanan authored
wgrad should be zero'ed out if a weight parameter is shared among multiple layers Signed-off-by:Deepak Narayanan <dnarayanan@nvidia.com>
-
zlsh80826 authored
Support layernorm sm_margin through environment variables Signed-off-by: -
Ming-Xu Huang authored
Use relative idx to ScaledUpperTriangMaskedSoftmaxFwdPrimitive.abstract to support batching. Signed-off-by:Ming Huang <mingh@nvidia.com>
-
- 28 Nov, 2023 3 commits
-
-
Marks101 authored
* [PyTorch] Linear: fix computation for wgrad if sequence_parallel=True Signed-off-by:
-
Deepak Narayanan authored
Getting warnings of the following form using ToT TE: ``` /usr/local/lib/python3.10/dist-packages/transformer_engine/pytorch/attention.py:852: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() data_ptr = grad_outputs[0].storage().data_ptr() ``` Signed-off-by:Deepak Narayanan <2724038+deepakn94@users.noreply.github.com>
-
Tim Moon authored
Use unsigned char for byte dtype Signed-off-by:
-
- 23 Nov, 2023 1 commit
-
-
Tian Zheng authored
* Add async TP comm Signed-off-by:
Tian Zheng (Engrg-Hardware 1) <tizheng@nvidia.com> * Add set CUDA_DEVICE_MAX_CONNECTIONS warning for tp overlap Signed-off-by:
-
- 21 Nov, 2023 1 commit
-
-
Shijie authored
* fix cudnn FA softmax shape Signed-off-by:
Shijie Wang <jaywan@nvidia.com> * set inplace rng_state Signed-off-by:
-
- 20 Nov, 2023 1 commit
-
-
zlsh80826 authored
* Remove assertion for NO_MASK Signed-off-by:
-
- 17 Nov, 2023 3 commits
-
-
Przemek Tredak authored
Signed-off-by: -
cyanguwa authored
* disable FAv2.1 if causal+cross attn Signed-off-by:
-
Kirthi Shankar Sivamani authored
* Delay caching of transposes Signed-off-by:
-
- 16 Nov, 2023 1 commit
-
-
jon-chuang authored
add comments Signed-off-by:Jon Chuang <9093549+jon-chuang@users.noreply.github.com>
-
- 15 Nov, 2023 1 commit
-
-
cyanguwa authored
* fix condition checks related to FA head_dim Signed-off-by:
-
- 14 Nov, 2023 1 commit
-
-
Ming-Xu Huang authored
* Refactor sharding.py for the further custom_partitioning migration Signed-off-by:
Donglin Yang <dongliny@nvidia.com>
-
- 13 Nov, 2023 3 commits
-
-
Santosh Bhavani authored
* Update README.rst - Installation section Added pip install instructions and cleaned up pre-reqs and FlashAttention-2 section Signed-off-by:
-
Kirthi Shankar Sivamani authored
Improve PyTorch memory usage Signed-off-by: -
zlsh80826 authored
[C/JAX] Support more mask types for the arbitrary seqlen kernels and minor changes of JAX bias (#469) * Move bias to float32 Signed-off-by:
-
- 09 Nov, 2023 1 commit
-
-
Sangkug Lym authored
* Make user buffer name configurable Signed-off-by:
Sangkug Lym <slym@nvidia.com> * Apply suggestions from code review Signed-off-by:
-
- 08 Nov, 2023 2 commits
-
-
Selvaraj Anandaraj authored
* Returning an empty tensor of param dtype for wgrad Signed-off-by:
Selvaraj Anandaraj <selvaraja@computelab-frontend-4-ub22.nvidia.com> * lint Signed-off-by:
-
zlsh80826 authored
* Deprecate QKV_INTERLEAVED use in JAX Signed-off-by:
-
