"tests/git@developer.sourcefind.cn:tsoc/superbenchmark.git" did not exist on "6ecded2002a1f1a874466422407100120ae4143a"
- 15 Jan, 2025 1 commit
-
-
guyueh1 authored
* Add a compile option to compile activation kernels with fast math Signed-off-by:
Guyue Huang <guyueh@nvidia.com> * Fix Signed-off-by:
Kirthi Shankar Sivamani <ksivamani@nvidia.com> Signed-off-by:
guyueh1 <140554423+guyueh1@users.noreply.github.com> --------- Signed-off-by:
-
- 13 Jan, 2025 2 commits
-
-
Alp Dener authored
* support AG overlap in sequence-parallel Linear forward and RS overlap in sequence-parallel Linear backward Signed-off-by:
Alp Dener <adener@nvidia.com> * implemented TP overlap support for column-parallel te.Linear Signed-off-by:
pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
-
Sergii Dymchenko authored
Signed-off-by:Sergii Dymchenko <sdym@meta.com>
-
- 11 Jan, 2025 1 commit
-
-
Phuong Nguyen authored
* add test_multiprocessing_encoder with processing spawning in bash --------- Signed-off-by:Phuong Nguyen <phuonguyen@nvidia.com>
-
- 10 Jan, 2025 1 commit
-
-
Xiaowei Ren authored
Take token count quantization of fused attention into consideration for CP results correction (#1396) * fix second half lse shape Signed-off-by:
Xiaowei Ren <xren@nvidia.com> * bug fixes Signed-off-by:
-
- 08 Jan, 2025 4 commits
-
-
Xiaowei Ren authored
* make pad_between_seqs check do not consider padding at the end Signed-off-by:
-
Michael Goldfarb authored
Correct fused attention output after each step to reduce intermediate memory use. Signed-off-by:Michael Goldfarb <mgoldfarb@nvidia.com>
-
Liyuan Liu authored
the current implementation would release the output of ln, leading to an error if setting `return_layernorm_output=True`. Signed-off-by:
Liyuan Liu <llychinalz@gmail.com> Co-authored-by:
Tim Moon <4406448+timmoon10@users.noreply.github.com>
-
Reese Wang authored
* Fix SWA mask for THD and forcing seqlen_kv >= seqlen_q for SWA Signed-off-by:
Reese Wang <rewang@nvidia.com> * Generalize sliding window mask Signed-off-by:
-
- 02 Jan, 2025 1 commit
-
-
Kirthi Shankar Sivamani authored
Signed-off-by:
-
- 20 Dec, 2024 1 commit
-
-
Charlene Yang authored
* add swa (left,0) + padding + brcm support Signed-off-by:
Charlene Yang <8636796+cyanguwa@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * final fixes Signed-off-by:
-
- 18 Dec, 2024 3 commits
-
-
Phuong Nguyen authored
* Move test distributed encoder to L0 distributed test suit --------- Signed-off-by:
-
Charlene Yang authored
* WIP: fix get_swa_mask for padding Signed-off-by:
-
Charlene Yang authored
add weights_only=False for torch.load Signed-off-by:
-
- 17 Dec, 2024 2 commits
-
-
Reese Wang authored
* Add util functions to attn_mask_type Signed-off-by:
-
Charlene Yang authored
add max_t for KV Signed-off-by:
-
- 16 Dec, 2024 1 commit
-
-
Youngeun Kwon authored
* draft implementation of fsdp2 fp8 all gather Signed-off-by:
Youngeun Kwon <youngeunk@nvidia.com> * fix the convergence issue Signed-off-by:
-
- 14 Dec, 2024 2 commits
-
-
Phuong Nguyen authored
* softmax custom calls with correct encapsulates * rm jax deprecated features --------- Signed-off-by: -
Jingyue Wu authored
-
- 12 Dec, 2024 2 commits
-
-
Kirthi Shankar Sivamani authored
Add Jeremy to ci users Signed-off-by: -
Phuong Nguyen authored
* fix ctx.aval_out indexing for workspace * add cudnn init to prepare phase of norm custom calls * add thread_local for norm registry instance --------- Signed-off-by:
-
- 10 Dec, 2024 1 commit
-
-
Reese Wang authored
* Bug Fix: Use default factory for not sharing mutable default values --------- Signed-off-by:
-
- 06 Dec, 2024 2 commits
-
-
Phuong Nguyen authored
* cuDNN normalization integration * TE Norm refactor * TE Norm APIs changes. --------- Signed-off-by:
-
Tim Moon authored
Debug Mcore integration test Avoid FP8 on Ampere and older. Generate synthetic data instead of depending on external data. Signed-off-by:Tim Moon <tmoon@nvidia.com>
-
- 05 Dec, 2024 3 commits
-
-
Xiaowei Ren authored
* always have padding mask type for both flash and fused attentions Signed-off-by:
-
Tim Moon authored
Store module extra state in tensor Signed-off-by: -
Tim Moon authored
Debug jobs to deploy nightly docs Signed-off-by:
-
- 04 Dec, 2024 1 commit
-
-
Michael Goldfarb authored
Scale sequence length in CP tests to avoid tiny sizes. Signed-off-by:
-
- 02 Dec, 2024 2 commits
-
-
Youngeun Kwon authored
* draft implementation Signed-off-by:
-
Tim Moon authored
* Update list of CI users Signed-off-by:
-
- 27 Nov, 2024 1 commit
-
-
Xiaowei Ren authored
* retain_graph=True for grouped gemm Signed-off-by:
-
- 25 Nov, 2024 2 commits
-
-
Michael Goldfarb authored
Moved framework agnostic THD kernels to common. --------- Signed-off-by: -
buptzyb authored
* Align RNG tracker with megatron Signed-off-by:
Robin Zhang <robinz@nvidia.com> Co-authored-by:
Yifei Song <yifeis@nvidia.com> * Fix module_params order and warmup bug in cudagraph Signed-off-by:
Xin Yao <yaox12@outlook.com> * Revert CudaRNGStatesTracker Signed-off-by:
Xin Yao <xiny@nvidia.com> Co-authored-by:
-
- 22 Nov, 2024 1 commit
-
-
Tim Moon authored
* Add helper function to convert C++ container to string Signed-off-by:
-
- 21 Nov, 2024 1 commit
-
-
Tim Moon authored
* Handle deprecated `hidden_size` arg in norm modules Signed-off-by:
-
- 20 Nov, 2024 1 commit
-
-
Charlene Yang authored
* fix GQA error message Signed-off-by:
-
- 15 Nov, 2024 3 commits
-
-
Kenichi Maehashi authored
use CMAKE_CURRENT_SOURCE_DIR instead of CMAKE_SOURCE_DIR Signed-off-by:Kenichi Maehashi <webmaster@kenichimaehashi.com>
-
Przemek Tredak authored
Signed-off-by:Przemek Tredak <ptredak@nvidia.com>
-
Tim Moon authored
* Add activation ops Signed-off-by:
-
- 14 Nov, 2024 1 commit
-
-
Kirthi Shankar Sivamani authored
* Limit to one call of ctx.saved_tensors per autograd bwd Signed-off-by:
-
