- 20 May, 2025 2 commits
- 14 May, 2025 1 commit
-
-
wenjh authored
Add rules of cuda_runtime.h, cuda_driver.h and cuda_nvml.h to hip. Signed-off-by:wenjh <wenjh@sugon.com>
-
- 13 May, 2025 1 commit
-
-
yuguo authored
-
- 08 May, 2025 1 commit
-
-
yuguo authored
-
- 07 May, 2025 2 commits
- 06 May, 2025 2 commits
-
-
yuguo authored
-
wenjh authored
Fix launch bounds of multi_tensor_apply_kernel and thd_out_correction_kernel. Signed-off-by:
-
- 29 Apr, 2025 3 commits
-
-
wenjh authored
-
yuguo authored
-
wenjh authored
Signed-off-by:
-
- 25 Apr, 2025 2 commits
-
-
yuguo authored
-
panning authored
API `rmsnorm_forward` of python returns 3 values rather than 2 from V2.3 Signed-off-by:
-
- 18 Apr, 2025 2 commits
-
-
yuguo authored
-
Hongbin Liu authored
* split wgrad for GroupedLinear Signed-off-by:
Hongbin Liu <hongbinl@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by:
Kirthi Shankar Sivamani <ksivamani@nvidia.com> * Update transformer_engine/pytorch/module/linear.py Signed-off-by:
-
- 17 Apr, 2025 4 commits
-
-
wdykas authored
* re merge request Signed-off-by:
Peter Dykas <wdykas@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add docstring Signed-off-by:
-
Xin Yao authored
* move swizzle scaling factor to cpp Signed-off-by:
Xin Yao <xiny@nvidia.com> * resolve comments Signed-off-by:
pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
-
kwyss-nvidia authored
* Allow NVTEShape to own data. Signed-off-by:
Keith Wyss <kwyss@nvidia.com> * Convert repeated copy paths to nvte_make_shape calls. Signed-off-by:
Tim Moon <4406448+timmoon10@users.noreply.github.com> * Build fixes. Signed-off-by:
-
Paweł Gadziński authored
* drop Signed-off-by:
Pawel Gadzinski <pgadzinski@nvidia.com> * fix Signed-off-by:
-
- 16 Apr, 2025 1 commit
-
-
Paweł Gadziński authored
* add Signed-off-by:
Przemyslaw Tredak <ptrendx@gmail.com> Signed-off-by:
Paweł Gadziński <62263673+pggPL@users.noreply.github.com> * setup fix Signed-off-by:
-
- 15 Apr, 2025 2 commits
-
-
Li Tao authored
* support adam bf16 state Signed-off-by:
XiaobingSuper <xiaobingzhangupc@gmail.com> * use fp32 kernel but keep bf16 optimizer states to save memory Signed-off-by:
lit <lit@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by:
-
Paweł Gadziński authored
* fix Signed-off-by:
-
- 14 Apr, 2025 6 commits
-
-
Jianbin Chang authored
* Add fp8 weight transpose cache check in backward, and regenerated it if it does not exist Signed-off-by:
jianbinc <shjwudp@gmail.com> * Properly handle fsdp shard model weight input. Signed-off-by:
-
Tim Moon authored
* Avoid unnecessary tensor usages when caching for linear op backward Signed-off-by:
Tim Moon <tmoon@nvidia.com> * Debug test failure Signed-off-by:
-
Xin Yao authored
* Enable MXFP8 and Per-Tensor Current Scaling for Grouped Linear Signed-off-by:
-
yuguo authored
-
Autumn1998 authored
* add support for new recipe on permute_fusion, rm fp unpermute Signed-off-by:
tongliu <tongliu@nvidia.com> * fix lint Signed-off-by:
-
Selvaraj Anandaraj authored
* Added attention activation offloading support for TE v2.0 Signed-off-by:
Selvaraj Anandaraj <selvaraja@login-ptyche02.ptyche.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by:
-
- 11 Apr, 2025 2 commits
-
-
Tim Moon authored
* Add option to cache activation input in FP8 Signed-off-by:
-
yuguo authored
-
- 10 Apr, 2025 1 commit
-
-
kwyss-nvidia authored
* Add GEMM logic for blockwise quantized tensors. GEMM test cases included in pytorch integration. Signed-off-by:
zhongboz <zhongboz@nvidia.com> * Edit and cleanup BF16 gather code. Signed-off-by:
Xin Yao <yaox12@outlook.com> * Update test_float8blockwisetensor.py Signed-off-by:
-
- 09 Apr, 2025 2 commits
-
-
Tim Moon authored
* Debug checkpointing with te.Sequential Signed-off-by:
-
Tim Moon authored
Explicitly specify quantized tensor usages needed for linear op backward Signed-off-by:
-
- 08 Apr, 2025 2 commits
-
-
vasunvidia authored
* Use dummy wgrads for lower memory consumption Signed-off-by:
Vasudevan Rengasamy <vrengasamy@nvidia.com> * Bug fix to avoid sharing gradients. Signed-off-by:
-
panning authored
-
- 07 Apr, 2025 3 commits
-
-
Jianbin Chang authored
Support fp8 primary weight in fsdp training Signed-off-by:
-
Xin Yao authored
* fix cpp warning Signed-off-by:
-
Xin Yao authored
* refactor to add cp support for sbhd/bshd Signed-off-by:
-
- 04 Apr, 2025 1 commit
-
-
kwyss-nvidia authored
* Blockwise float8 quantizer and quantized tensor class. The classes are configurable for 128x128 blocksize and 1x128 blocksize via setting block_scaling_dim == 2,1 respectively. Scale tensors are stored in a format emenable for matrix multiplication, however the integration of matmul is deferred as a separate story. Fusions of quantization and DBIAS or activation functions are not yet implemented, and the dequantization is currently implemented in torch. Tests for quantization are included in C++ and pytorch layers, with exact comparison to reference quantizer behavior as well as an attempt to hit interesting branches through the API such as tensor creation in pytorch and CPP and dequantization of row and columnwise usage. Two CUDA kernels for quantization are included, and are direct ports of equivalents in the kitchen repository, where a subchannel recipe has been used for end to end training. Signed-off-by:
-
