- 25 Oct, 2024 3 commits
-
-
Charlene Yang authored
* WIP: add max_t support for THD Signed-off-by:
Charlene Yang <8636796+cyanguwa@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * WIP: save tensors for debug and point to new FE Signed-off-by:
pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
-
Charlene Yang authored
* add THD MQA/GQA Signed-off-by:
-
Phuong Nguyen authored
Update jax version for ffi Signed-off-by:Phuong Nguyen <phuonguyen@nvidia.com>
-
- 24 Oct, 2024 3 commits
-
-
Tim Moon authored
Update class names for Paddle 3.0 Signed-off-by:Tim Moon <tmoon@nvidia.com>
-
Hua Huang authored
[JAX] XLA Custom Calls with FFI for FusedAttnFwd, Quantize, Transpose, ActLuFP8, LayerNormForwardFP8FFI, and LayerNormBackwardFFI (#1263) * Add TransposeFFI, test passed Signed-off-by:
Hua Huang <huah@nvidia.com> * Add ActLuFP8FFI; fix TransposeFFI Signed-off-by:
-
Michael Goldfarb authored
[JAX] Fix correctness of JAX fused attention with CP and improve numerics check in unit tests (#1282) Fix correctness of JAX fused attention with CP. Signed-off-by:Michael Goldfarb <mgoldfarb@nvidia.com>
-
- 22 Oct, 2024 2 commits
-
-
Reese Wang authored
Add THD + GQA supports for cuDNN >= 9.6 Signed-off-by:Reese Wang <rewang@nvidia.com>
-
Michael Goldfarb authored
* Use 64-bit offsets for cuDNN 9.5+ * Align workspace tensors to 16B. * Fix bug where std::accumulate overflowed on large tensor shapes. * Only support 64-bit offsets on arbitrary sequence length fp16 backend. Signed-off-by:
-
- 17 Oct, 2024 4 commits
-
-
Xiaowei Ren authored
fix seq_dim in CP implementation Signed-off-by:Xiaowei Ren <xren@nvidia.com>
-
Phuong Nguyen authored
* register CmdBufferCompatible traits via C++ API * renamed FFI_Traits * use register_ffi_target() --------- Signed-off-by: -
Xin Yao authored
* fix bias for 0-dim tensor Signed-off-by:
Xin Yao <xiny@nvidia.com> * add check Signed-off-by:
-
Xin Yao authored
Fix wgrad for GroupedLinear when weights doesn't require grad Signed-off-by:
Kirthi Shankar Sivamani <ksivamani@nvidia.com>
-
- 16 Oct, 2024 5 commits
-
-
Kirthi Shankar Sivamani authored
Fix FP8 activation recompute Signed-off-by: -
Kirthi Shankar Sivamani authored
* Upgrade pylint and first round formatting Signed-off-by:
-
Charlene Yang authored
* WIP: make FA2 optional Signed-off-by:
-
Md Fahim Faysal Khan authored
fixed assertion bug for SWA Signed-off-by:
Md Fahim Faysal Khan <mdfahimfaysa@nvidia.com> Co-authored-by:
Phuong Nguyen <36155692+phu0ngng@users.noreply.github.com>
-
Tim Moon authored
* Build custom ORT ops before running ONNX tests Signed-off-by:
-
- 15 Oct, 2024 1 commit
-
-
Michael Goldfarb authored
Update test to check support for context parallel attention. Signed-off-by:
-
- 14 Oct, 2024 1 commit
-
-
Tim Moon authored
Signed-off-by:
-
- 12 Oct, 2024 1 commit
-
-
Xin Yao authored
* Let Fused RoPE support THD with CP Signed-off-by:
Xiaowei Ren <103958965+xrennvidia@users.noreply.github.com>
-
- 11 Oct, 2024 2 commits
-
-
Xiaowei Ren authored
* fa2 function import renaming Signed-off-by:
-
李金梁 authored
* Fix bug in torch compile and seqdim is integer Signed-off-by:
李金梁 <975761915@qq.com> * Update attention.py change the jit_fuser to torch.compile on flash_attn_fwd_out_correction Signed-off-by:
-
- 10 Oct, 2024 2 commits
-
-
Przemyslaw Tredak authored
* Fixes to Float8Tensor Signed-off-by:
Przemyslaw Tredak <ptredak@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by:
-
Hua Huang authored
* Expose JAX sliding window attn API Signed-off-by:
-
- 09 Oct, 2024 3 commits
-
-
Charlene Yang authored
* improve get_attention_backend logic Signed-off-by:
-
Charlene Yang authored
* add extra_state change description for different TE versions Signed-off-by:
-
Tim Moon authored
* Handle Float8Tensor when casting module dtype Keep data in Float8Tensor and only change nominal dtype. Monkey-patch PyTorch module casting functions to handle Float8Tensor. Add tests. Signed-off-by:
Tim Moon <4406448+timmoon10@users.noreply.github.com> --------- Signed-off-by:
-
- 08 Oct, 2024 1 commit
-
-
Charlene Yang authored
* add qkv descales to FA3 Signed-off-by:
-
- 07 Oct, 2024 2 commits
-
-
Charlene Yang authored
* adjust window size to (i-window_size_left,i] for cuDNN Signed-off-by:
-
Xiaowei Ren authored
* change API for hierarchical CP Signed-off-by:
-
- 04 Oct, 2024 1 commit
-
-
Tim Moon authored
* CPU perf optimization in linear autograd function Avoid enable_grad context when possible in cast function. Cache distributed group properties. Signed-off-by:
-
- 03 Oct, 2024 1 commit
-
-
Charlene Yang authored
move block_table arg to varlen_func section Signed-off-by:
-
- 01 Oct, 2024 2 commits
-
-
Przemyslaw Tredak authored
* Removing the unused options from GroupedLinear docs and fixing the bug with offsets Signed-off-by:
-
Kirthi Shankar Sivamani authored
Add pool argument to make_graphed_callable Signed-off-by:
-
- 27 Sep, 2024 2 commits
-
-
Charlene Yang authored
* fix detection of 3 in 3hd/h3d layouts Signed-off-by:
-
Paweł Gadziński authored
* Docs fixes Signed-off-by:
Pawel Gadzinski <pgadzinski@nvidia.com> * docs fix Signed-off-by:
-
- 20 Sep, 2024 1 commit
-
-
Przemyslaw Tredak authored
* Restore compatibility with Python 3.8 Signed-off-by:
-
- 19 Sep, 2024 1 commit
-
-
Xin Yao authored
* relax contiguous check for flash attention Signed-off-by:
-
- 18 Sep, 2024 2 commits
-
-
Sudhakar Singh authored
* make rotary_base arg Signed-off-by:
Sudhakar Singh <sudhakars@nvidia.com> * rotary base can be a float Co-authored-by:
-
Alp Dener authored
* Check if network interface name is valid and show useful warning message when initializing Userbuffers Signed-off-by:
Alp Dener <adener@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix formatting issue in warning message. Co-authored-by:
-
