- 23 Sep, 2023 1 commit
-
-
cyanguwa authored
* [PyTorch] Implement GQA based on fused q, k, v projection. Additionally fixes #392 Signed-off-by:
Markus Schnoes <markus.schnoes@gmx.de> * [PyTorch] Extend parameters_split option in Linear and LayerNormLinear to support splitting with different sizes as required by unfused GQA. Signed-off-by:
Charlene Yang <8636796+cyanguwa@users.noreply.github.com> * fix noop cat to bypass torch.cat and support uneven split Signed-off-by:
-
- 22 Sep, 2023 1 commit
-
-
Kirthi Shankar Sivamani authored
Fix for sequence-parallel Signed-off-by:Kirthi Shankar Sivamani <ksivamani@nvidia.com>
-
- 20 Sep, 2023 1 commit
-
-
Przemyslaw Tredak authored
* Enable the model to be change precision between iterations Signed-off-by:
Przemek Tredak <ptredak@nvidia.com> * Add test Signed-off-by:
-
- 18 Aug, 2023 1 commit
-
-
Sudhakar Singh authored
Signed-off-by:Sudhakar Singh <sudhakars@nvidia.com>
-
- 16 Aug, 2023 2 commits
-
-
Kirthi Shankar Sivamani authored
* CPU initialization Signed-off-by:
-
Kirthi Shankar Sivamani authored
* Initial refactor Signed-off-by:
-
- 10 Aug, 2023 2 commits
-
-
Kirthi Shankar Sivamani authored
Signed-off-by: -
Jason Wang authored
* fix linear sequence parallel when weight is frozen Signed-off-by:
jasonwan <jasonwan@nvidia.com> * Extend fix to all APIs Signed-off-by:
-
- 08 Aug, 2023 2 commits
-
-
Kirthi Shankar Sivamani authored
* Optimize calls to .cpu() during checkpointing Signed-off-by:
-
Przemyslaw Tredak authored
Fix for the RMSNorm tests/doc/ONNX export to match the actual implementation Signed-off-by:
-
- 02 Aug, 2023 1 commit
-
-
Kirthi Shankar Sivamani authored
Signed-off-by:
-
- 28 Jul, 2023 1 commit
-
-
Kirthi Shankar Sivamani authored
Minor bug fix Signed-off-by:
-
- 27 Jul, 2023 1 commit
-
-
Przemyslaw Tredak authored
* Exposing RMSNorm in pyTorch extensions Signed-off-by:
Przemyslaw Tredak <ptrendx@gmail.com> Signed-off-by:
-
- 15 Jul, 2023 1 commit
-
-
Tim Moon authored
* Disable TorchDynamo optimizations in PyTorch modules Signed-off-by:
Tim Moon <tmoon@nvidia.com> * Add test for Torch Dynamo Signed-off-by:
Tim Moon <4406448+timmoon10@users.noreply.github.com> Signed-off-by:
-
- 14 Jul, 2023 2 commits
-
-
Kirthi Shankar Sivamani authored
Bug fix for checkpointing Signed-off-by: -
Kirthi Shankar Sivamani authored
* Deprecate unused APIs Signed-off-by:
-
- 13 Jul, 2023 3 commits
-
-
Neta Zmora authored
* Fix FP32 LayerNorm ONNX export When running inference use a fwd method that is registered with torchscript. Signed-off-by:
Neta Zmora <nzmora@nvidia.com> * Bug fix Signed-off-by:
-
Kirthi Shankar Sivamani authored
Remove extra buffers Signed-off-by: -
Kirthi Shankar Sivamani authored
* Better dimension assert for FP8 Signed-off-by:
-
- 26 Jun, 2023 1 commit
-
-
Kirthi Shankar Sivamani authored
* Get default dtype from pytorch Signed-off-by:
-
- 20 Jun, 2023 1 commit
-
-
Kirthi Shankar Sivamani authored
Signed-off-by:
-
- 13 Jun, 2023 1 commit
-
-
Przemyslaw Tredak authored
* Added ReLU and GLU variants to common Signed-off-by:
-
- 02 Jun, 2023 1 commit
-
-
Jan Bielak authored
* Ignore IDE files Signed-off-by:
Jan Bielak <jbielak@nvidia.com> * Fix typing errors Signed-off-by:
-
- 01 Jun, 2023 1 commit
-
-
Sudhakar Singh authored
* extend fp8 weight placeholders logic for Linear, LNLinear, LNMLP Signed-off-by:
-
- 25 May, 2023 1 commit
-
-
Carlos Mocholí authored
* Clearer error messages for dtype and shape assertions Signed-off-by:
Carlos Mocholí <carlossmocholi@gmail.com> * Update transformer_engine/pytorch/utils.py Co-authored-by:
-
- 09 May, 2023 1 commit
-
-
Kirthi Shankar Sivamani authored
* Initial refactor Signed-off-by:
-
