- 25 Jun, 2024 1 commit
-
-
Xin Yao authored
* GroupedGEMM via multi-stream cublas * fix A/B is nullptr while D is not nullptr * add fp8 grouped gemm * register with TorchScript * add the GroupedLinear layer --------- Signed-off-by:
Xin Yao <xiny@nvidia.com> Signed-off-by:
Phuong Nguyen <phuonguyen@nvidia.com> Co-authored-by:
Jiang Shao <jiangs@nvidia.com> Co-authored-by:
Qi Zhang <qizhang@nvidia.com> Co-authored-by:
-
- 18 Jun, 2024 4 commits
-
-
Tim Moon authored
Release GIL in PyTorch pybind11 functions Signed-off-by:Tim Moon <tmoon@nvidia.com>
-
Charlene Yang authored
* simplify offset tensors Signed-off-by:
Charlene Yang <8636796+cyanguwa@users.noreply.github.com> * minor fixes; tests pass Signed-off-by:
pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
-
Charlene Yang authored
fix tp_initialized error Signed-off-by: -
Kirthi Shankar Sivamani authored
* Remove optional UB build leftovers Signed-off-by:
Kirthi Shankar Sivamani <ksivamani@nvidia.com> * rm unused import Signed-off-by:
-
- 17 Jun, 2024 1 commit
-
-
Sangkug Lym authored
* Add the option to use SM for P2P comm in TP overlap Signed-off-by:
Sangkug Lym <slym@nvidia.com> * cleanup Signed-off-by:
Tim Moon <4406448+timmoon10@users.noreply.github.com> --------- Signed-off-by:
-
- 15 Jun, 2024 1 commit
-
-
Charlene Yang authored
* subclass DPA with BaseModule and test with test_gpt_checkpointing Signed-off-by:
-
- 14 Jun, 2024 4 commits
-
-
Pavel Shamis (Pasha) authored
* A hot fix to disable CE deadlock check Signed-off-by:
Pavel Shamis (Pasha) <pasharesearch@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by:
-
Kirthi Shankar Sivamani authored
* Apply formatting Signed-off-by:
-
Kirthi Shankar Sivamani authored
* Initial config test Signed-off-by:
-
Charlene Yang authored
* add attention docs Signed-off-by:
-
- 13 Jun, 2024 4 commits
-
-
Alp Dener authored
* added DL framework callbacks for bootstrapping userbuffers without MPI Signed-off-by:
Alp Dener <adener@nvidia.com> * removed userbuffers availability check in TE modules since userbuffers is now always compiled Signed-off-by:
-
BoxiangW authored
* Add norm_factor arg into DotProductAttention Signed-off-by:
Boxiang Wang <boxiangw@nvidia.com> * Change kwarg name from `norm_factor` to `softmax_scale` Signed-off-by:
BoxiangW <45734921+BoxiangW@users.noreply.github.com> --------- Signed-off-by:
-
Alp Dener authored
reverting autocast API back to PyTorch v2.3.1 and below Signed-off-by: -
Xin Yao authored
* expose multi_tensor_* kernels Signed-off-by:
-
- 12 Jun, 2024 3 commits
-
-
Sudhakar Singh authored
skip switching to nvfuser for torch >= 2.2 Signed-off-by:Sudhakar Singh <sudhakars@nvidia.com>
-
Alp Dener authored
added @torch._disable_dynamo fixed deprecation warnings with torch autocast API for TE checkpoint Signed-off-by: -
Alp Dener authored
restricted fsdp asserts on primary fp8 weights to TE modules Signed-off-by:
-
- 10 Jun, 2024 2 commits
-
-
Xiaowei Ren authored
* add seq_offsets_qkvo for cudnn thd Signed-off-by:
Xiaowei Ren <xren@nvidia.com> * add seq_offsets_qkvo to AttnFuncWithCP Signed-off-by:
-
Tim Moon authored
* Avoid select operation in cast-transpose extension Signed-off-by:
-
- 07 Jun, 2024 2 commits
-
-
Alp Dener authored
* New TE wrapper for PyTorch FullyShardedDataParallel to make TE modules distribute their activations after the forward pass and gather them before the backward pass Signed-off-by:
-
Kirthi Shankar Sivamani authored
* Remove interval arg from recipe Signed-off-by:
-
- 06 Jun, 2024 1 commit
-
-
Kirthi Shankar Sivamani authored
Cleanup Signed-off-by:
-
- 03 Jun, 2024 1 commit
-
-
Tim Moon authored
* Modify CUDA graph tests to use grad accumulation steps Signed-off-by:
-
- 31 May, 2024 1 commit
-
-
Tim Moon authored
Replace int8_t in PyTorch extensions with int64_t Signed-off-by:
-
- 30 May, 2024 3 commits
-
-
Charlene Yang authored
* add THD support Signed-off-by:
-
Xin Yao authored
* add multi-tensor kernels Signed-off-by:
-
Tim Moon authored
* Initial refactor of FP8 workspaces in Linear module Signed-off-by:
-
- 29 May, 2024 1 commit
-
-
Tim Moon authored
Make sure RoPE frequencies are in FP32 Signed-off-by:
-
- 25 May, 2024 1 commit
-
-
Paweł Gadziński authored
* Fixed Llama tutorial. Changed batch size and added fused=True. Signed-off-by:
Pawel Gadzinski <pgadzinski@nvidia.com> Signed-off-by:
root <root@ipp2-0037.nvidia.com> * Tutorial updated but not complete yet. Signed-off-by:
root <root@ipp2-1661.nvidia.com> * Comments Signed-off-by:
root <root@ipp2-2373.nvidia.com> Co-authored-by:
root <root@ipp2-1588.nvidia.com> Co-authored-by:
root <root@ipp2-2371.nvidia.com> Co-authored-by:
root <root@ipp2-1589.nvidia.com> Co-authored-by:
-
- 22 May, 2024 1 commit
-
-
Alp Dener authored
TE checkpoint now preserves the torch autocast context from the forward pass during the recompute phase Signed-off-by:
-
- 21 May, 2024 2 commits
-
-
Alp Dener authored
replaced deprecated pkg_resources with packaging Signed-off-by: -
Pavel Shamis (Pasha) authored
-
- 20 May, 2024 1 commit
-
-
Paweł Gadziński authored
* Calibration fix Signed-off-by:
-
- 17 May, 2024 1 commit
-
-
Charlene Yang authored
* fix inconsistency for attn mask; now True means participating in attn Signed-off-by:
-
- 16 May, 2024 1 commit
-
-
Phuong Nguyen authored
* added squared relu in te-torch Signed-off-by:
-
- 13 May, 2024 1 commit
-
-
Kunlun Li authored
Signed-off-by:
kunlunl <kunlunl@nvidia.com> Co-authored-by:
-
- 09 May, 2024 1 commit
-
-
Kirthi Shankar Sivamani authored
Bump FA version to 2.5.8 Signed-off-by:
-
- 02 May, 2024 1 commit
-
-
cyanguwa authored
* initialize tp_group for FP8 DPA Signed-off-by:
-
- 01 May, 2024 1 commit
-
-
Jinze Xue authored
* Handle the scaling factor when amax is too tiny that leads to an infinite scale Signed-off-by:
Jinze Xue <jinzex@nvidia.com> * revert formatting changes Signed-off-by:
Jinze Xue <155670984+jinzex@users.noreply.github.com> * Apply review suggestion Co-authored-by:
-
