"transformer_engine/pytorch/ops/fuser.py" did not exist on "05977f448f3735499dcaaff47d47777d8e89105e"
- 12 Jun, 2024 1 commit
-
-
Alp Dener authored
restricted fsdp asserts on primary fp8 weights to TE modules Signed-off-by:Alp Dener <adener@nvidia.com>
-
- 11 Jun, 2024 2 commits
-
-
Keshav Balasubramanian authored
Signed-off-by:Keshav <keshavb@nvidia.com>
-
Phuong Nguyen authored
* added distributed test for ln_mlp primitive * added distributed test for LayerNorm layer * changed error messages --------- Signed-off-by:Phuong Nguyen <phuonguyen@nvidia.com>
-
- 10 Jun, 2024 4 commits
-
-
Phuong Nguyen authored
- Made order of gated act consistent in all branches Signed-off-by: -
Xiaowei Ren authored
* add seq_offsets_qkvo for cudnn thd Signed-off-by:
Xiaowei Ren <xren@nvidia.com> * add seq_offsets_qkvo to AttnFuncWithCP Signed-off-by:
Charlene Yang <8636796+cyanguwa@users.noreply.github.com>
-
Tim Moon authored
* Avoid select operation in cast-transpose extension Signed-off-by:
Tim Moon <tmoon@nvidia.com> * Avoid select operation in cast-transpose-dbias extensions Signed-off-by:
-
Kirthi Shankar Sivamani authored
Make transformer_engine::getenv independent of C++ ABI version Signed-off-by:Kirthi Shankar Sivamani <ksivamani@nvidia.com>
-
- 08 Jun, 2024 1 commit
-
-
Phuong Nguyen authored
* categorized `csrc/modules.cpp` Signed-off-by:
-
- 07 Jun, 2024 5 commits
-
-
Tim Moon authored
Fix bug when triggering CI Signed-off-by: -
Alp Dener authored
* New TE wrapper for PyTorch FullyShardedDataParallel to make TE modules distribute their activations after the forward pass and gather them before the backward pass Signed-off-by:
-
Kirthi Shankar Sivamani authored
Fix editable build for paddle Signed-off-by: -
Kirthi Shankar Sivamani authored
Signed-off-by: -
Kirthi Shankar Sivamani authored
* Remove interval arg from recipe Signed-off-by:
-
- 06 Jun, 2024 1 commit
-
-
Kirthi Shankar Sivamani authored
Cleanup Signed-off-by:
-
- 05 Jun, 2024 1 commit
-
-
Oleg Goncharov authored
* Merged CT+dbias+dact into a single template Signed-off-by:
Oleg Goncharov <ogoncharov@nvidia.com> * Moved gated activations ifrom the cast_transpose_fused ito a sseparate cpp file Signed-off-by:
Tim Moon <4406448+timmoon10@users.noreply.github.com> Signed-off-by:
Oleg Goncharov <64355998+Oleg-Goncharov@users.noreply.github.com> * Update transformer_engine/common/transpose/cast_transpose_fusion.cu Co-authored-by:
-
- 03 Jun, 2024 1 commit
-
-
Tim Moon authored
* Modify CUDA graph tests to use grad accumulation steps Signed-off-by:
-
- 01 Jun, 2024 1 commit
-
-
Paweł Gadziński authored
* Llama 3 update Signed-off-by:
Pawel Gadzinski <pgadzinski@nvidia.com> * Times update Signed-off-by:
Sudhakar Singh <sudhakars@nvidia.com> * small fixes Signed-off-by:
-
- 31 May, 2024 1 commit
-
-
Tim Moon authored
Replace int8_t in PyTorch extensions with int64_t Signed-off-by:
-
- 30 May, 2024 3 commits
-
-
Charlene Yang authored
* add THD support Signed-off-by:
-
Xin Yao authored
* add multi-tensor kernels Signed-off-by:
Xin Yao <xiny@nvidia.com> * add FusedAdam Signed-off-by:
-
Tim Moon authored
* Initial refactor of FP8 workspaces in Linear module Signed-off-by:
-
- 29 May, 2024 2 commits
-
-
Przemyslaw Tredak authored
* Change the documentation footer Signed-off-by:
Przemek Tredak <ptredak@nvidia.com> * Update docs toolchain versions Signed-off-by:
-
Tim Moon authored
Make sure RoPE frequencies are in FP32 Signed-off-by:
-
- 28 May, 2024 2 commits
-
-
Tim Moon authored
* Use correct FP8 group in multi-GPU docs FP8 process group should be tensor-parallel group Signed-off-by:
-
Tim Moon authored
Signed-off-by:
-
- 25 May, 2024 2 commits
-
-
Charlene Yang authored
allow bias support for sm80/86/89 for cuDNN 9+ Signed-off-by: -
Paweł Gadziński authored
* Fixed Llama tutorial. Changed batch size and added fused=True. Signed-off-by:
root <root@ipp2-0037.nvidia.com> * Tutorial updated but not complete yet. Signed-off-by:
root <root@ipp2-1661.nvidia.com> * Comments Signed-off-by:
root <root@ipp2-2373.nvidia.com> Co-authored-by:
root <root@ipp2-1588.nvidia.com> Co-authored-by:
root <root@ipp2-2371.nvidia.com> Co-authored-by:
root <root@ipp2-1589.nvidia.com> Co-authored-by:
-
- 22 May, 2024 3 commits
-
-
Alp Dener authored
TE checkpoint now preserves the torch autocast context from the forward pass during the recompute phase Signed-off-by: -
Ming-Xu Huang authored
* Fixed the shape mismatching issue in MLP. Signed-off-by:
Ming Huang <mingh@nvidia.com> * Add a corresponding test Signed-off-by:
Phuong Nguyen <36155692+phu0ngng@users.noreply.github.com>
-
Phuong Nguyen authored
* added alignment requirements for CuBLAS heuristics Signed-off-by:
-
- 21 May, 2024 2 commits
-
-
Alp Dener authored
replaced deprecated pkg_resources with packaging Signed-off-by: -
Pavel Shamis (Pasha) authored
-
- 20 May, 2024 2 commits
-
-
Paweł Gadziński authored
* Calibration fix Signed-off-by:
-
Przemek Tredak authored
Signed-off-by:
-
- 17 May, 2024 2 commits
-
-
Charlene Yang authored
* fix inconsistency for attn mask; now True means participating in attn Signed-off-by:
-
Shijie authored
* support main_grad Signed-off-by:
Shijie Wang <jaywan@nvidia.com> * update main_grad and fuse_wgrad_accumulation Signed-off-by:
-
- 16 May, 2024 1 commit
-
-
Phuong Nguyen authored
* added squared relu in te-torch Signed-off-by:
-
- 15 May, 2024 2 commits
-
-
Ming-Xu Huang authored
Remove act_enum from the del list ActLuPrimitive.partition Signed-off-by:
-
Kirthi Shankar Sivamani authored
This reverts commit 07291027 . Signed-off-by:
-
- 14 May, 2024 1 commit
-
-
cyanguwa authored
* use 3rdparty cudnn-frontend cmake to find cuDNN Signed-off-by:
-
