- 24 Jan, 2024 2 commits
-
-
Marks101 authored
[PyTorch] fix forward attention_type in MultiheadAttention Signed-off-by:
Markus Schnoes <markus.schnoes@gmx.de> Co-authored-by:
Kirthi Shankar Sivamani <ksivamani@nvidia.com>
-
Alp Dener authored
[PyTorch] Workaround for incorrect output from torch.cuda.is_bf16_compatible() on V100s and TU102s (#626) * replaced torch.cuda.is_bf16_compatible() with explicit sm_80 check via torch.cuda.get_device_capability() Signed-off-by:
Alp Dener <adener@nvidia.com> * implement te.utils.is_bf16_compatible() to replace torch.cuda counterpart Signed-off-by:
-
- 23 Jan, 2024 1 commit
-
-
Alp Dener authored
* added missing parameter materialization on real device for LayerNorm and RMSNorm Signed-off-by:
-
- 22 Jan, 2024 1 commit
-
-
Marks101 authored
Signed-off-by:
-
- 21 Jan, 2024 1 commit
-
-
Selvaraj Anandaraj authored
Activation offloading to CPU's for the Linear, Layernorm Linear and the Layernorm MLP modules (#571) * Added support activation offloading to CPU's Signed-off-by:
Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> * Moving CPU offloading library to TE Signed-off-by:
Przemek Tredak <ptredak@nvidia.com> * Fix to the code block in docs Signed-off-by:
-
- 20 Jan, 2024 1 commit
-
-
Sudhakar Singh authored
fix failing tests due to PR #557 Signed-off-by:
Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by:
cyanguwa <8636796+cyanguwa@users.noreply.github.com>
-
- 19 Jan, 2024 2 commits
-
-
Kirthi Shankar Sivamani authored
Signed-off-by: -
Kirthi Shankar Sivamani authored
Signed-off-by:
-
- 18 Jan, 2024 1 commit
-
-
Sudhakar Singh authored
* make TransformerLayer accept a `bshd` or `sbhd` tensor format Signed-off-by:
-
- 17 Jan, 2024 1 commit
-
-
Alp Dener authored
* Implemented deferred initialization via `device='meta'` option for te.Linear and added new PyTorch example to demonstrate its use with FullyShardedDataParallel execution. Signed-off-by:
-
- 10 Jan, 2024 2 commits
-
-
Zhang Haitao authored
* support non-tensor inputs/outputs for checkpoint Signed-off-by:
skydoorkai <htsantaclara@163.com> * better format Signed-off-by:
-
Xiaowei Ren authored
* try to use cuDNN fused attention for context parallelism Signed-off-by:
xren <xren@nvidia.com> * assert CP is only supported with NVTE_F16_arbitrary_seqlen Signed-off-by:
-
- 08 Jan, 2024 2 commits
-
-
Tim Moon authored
* Refactor parameter split in Linear module Remove module state from noop_cat. Support arbitrary names in parameter split. Handle tensor parallelism. Signed-off-by:
Tim Moon <tmoon@nvidia.com> * Make noop_cat a standalone operation Signed-off-by:
-
Jaemin Choi authored
* Use jit_fuser for bias-dropout-add fusion Signed-off-by:
Jaemin Choi <jaeminc@nvidia.com> * Use jit_fuser for CP FA kernel Signed-off-by:
-
- 06 Jan, 2024 1 commit
-
-
Kirthi Shankar Sivamani authored
* Deterministic FA, bump minimum supported version Signed-off-by:
-
- 05 Jan, 2024 1 commit
-
-
Przemyslaw Tredak authored
Signed-off-by:
-
- 03 Jan, 2024 3 commits
-
-
Przemyslaw Tredak authored
Signed-off-by:
-
Sangkug Lym authored
* Provide pre-computed max sequence to remove unnecessary kernels and D2H copies Signed-off-by:
Sangkug Lym <slym@nvidia.com> * Tweak comments Signed-off-by:
Tim Moon <4406448+timmoon10@users.noreply.github.com> --------- Signed-off-by:
-
Przemyslaw Tredak authored
Signed-off-by:
-
- 02 Jan, 2024 1 commit
-
-
Hongbin Liu authored
avoid redundant computation for cu_seqlens Signed-off-by:
Hongbin Liu <hongbinl@nvidia.com> Co-authored-by:
-
- 18 Dec, 2023 1 commit
-
-
Alp Dener authored
* Linear and LayerNormLinear weight and bias buffer cleanup at the end of init when there is no parameter split Signed-off-by:
-
- 16 Dec, 2023 2 commits
-
-
cyanguwa authored
* add sliding window to FA Signed-off-by:
-
Tim Moon authored
* Update fp8_meta amax when copying into Float8Tensor Signed-off-by:
-
- 15 Dec, 2023 3 commits
-
-
Przemyslaw Tredak authored
* Disable dynamo for Fused Attention Signed-off-by:
-
Kunlun Li authored
Avoid redeclaration error of "nv_bfloat16" when Compiling in CUDA12.1. Error log: /usr/local/cuda/include/cuda_fp16.hpp(2724): error: invalid redeclaration of type name "nv_bfloat16" (declared at line 2837 of /usr/local/cuda/include/cuda_bf16.hpp) Signed-off-by:Kunlun Li <94586211+kunlunl@users.noreply.github.com>
-
Fabian Joswig authored
[fix] fixed micro batched inference with RoPE Signed-off-by:
Fabian Joswig <fabian.joswig@deepl.com> Co-authored-by:
-
- 13 Dec, 2023 1 commit
-
-
Marks101 authored
Signed-off-by:
-
- 12 Dec, 2023 2 commits
-
-
cyanguwa authored
disable pylint bare except Signed-off-by: -
cyanguwa authored
* fix onnx/dynamo error Signed-off-by:
Przemyslaw Tredak <ptrendx@gmail.com> Signed-off-by:
-
- 07 Dec, 2023 2 commits
-
-
cyanguwa authored
* Integrate cuDNN frontend v1 to fused attention and miscellaneous fixes Signed-off-by:
-
Tim Moon authored
* Float8Tensor uses cached transpose if available Signed-off-by:
-
- 04 Dec, 2023 1 commit
-
-
Marks101 authored
* [PyTorch] TransformerLayer: add parallel_attention_mlp to support Falcon models Signed-off-by:
-
- 01 Dec, 2023 2 commits
-
-
Tim Moon authored
Fix incorrect variable name in LayerNormMLP backward Signed-off-by: -
LadyRick authored
[PyTorch] fix amax calculate during fp8 calibration Signed-off-by:ladyrick <ladyrick@qq.com>
-
- 30 Nov, 2023 1 commit
-
-
Deepak Narayanan authored
wgrad should be zero'ed out if a weight parameter is shared among multiple layers Signed-off-by:Deepak Narayanan <dnarayanan@nvidia.com>
-
- 28 Nov, 2023 2 commits
-
-
Marks101 authored
* [PyTorch] Linear: fix computation for wgrad if sequence_parallel=True Signed-off-by:
-
Deepak Narayanan authored
Getting warnings of the following form using ToT TE: ``` /usr/local/lib/python3.10/dist-packages/transformer_engine/pytorch/attention.py:852: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() data_ptr = grad_outputs[0].storage().data_ptr() ``` Signed-off-by:Deepak Narayanan <2724038+deepakn94@users.noreply.github.com>
-
- 17 Nov, 2023 2 commits
-
-
cyanguwa authored
* disable FAv2.1 if causal+cross attn Signed-off-by:
-
Kirthi Shankar Sivamani authored
* Delay caching of transposes Signed-off-by:
-
- 15 Nov, 2023 1 commit
-
-
cyanguwa authored
* fix condition checks related to FA head_dim Signed-off-by:
-
