GitLab

fixing attention perf test attempt 1

Optimized CUDA kernels for S2 Attention (forward and backward)

Merge branch 'mr/bwd-channel-permute-experiments' of https://github.com/rietmann-nv/torch-harmonics into mr/bwd-channel-permute-experiments

Also: Made fwd kernel use modified memory layout with standard shape

Also match the gradient output to the input, in terms of memory layout

Detect memory layout (B,C,H,W) (stride for C should be 1, if not, fix it)

This ensures that the backwards kernel is fast

putting qy in shared is a little faster

Changing internal memory layout means we can leave code in standard shape and
only change layout external to kernel

Introduce new CUDA kernels, `s2_attention_bwd_dkvq_kernel_mbT` and
`s2_attention_kernel_mbT`, for more efficient computation of backward gradients
and forward attention respectively. These changes optimize memory access
patterns and employ coalesced operations by leveraging tensor transpositions.

Forward kernel written by Mauro Bisson
Backwards kernel written by Andrea Paris (aparis@ethz.ch) and Max Rietmann

Parallelization strategy computes 1 output per Warp, with threads computing the
dot-product in parallel. Because inputs are transposed to have channel dimension
last, the dot-product memory access pattern is perfectly coalesced, leading to
excellent performance. This is true across both forward and backward kernels.
Co-authored-by: Mauro Bisson <maurob@nvidia.com>
Co-authored-by: Max Rietmann <mrietmann@nvidia.com>
Co-authored-by: Andrea Paris <aparis@ethz.ch>

fixing bug in quadrature weights for full attention. Adding better unit tests for attention. Cleanup in the cuda code.

* setting imaginary parts of DCT and nyquist frequency to zero in IRSHT variants

This reverts commit 82881276.

* setting imaginary parts of DCT and nyquist frequency to zero in IRSHT variants

* small fix

* making einsum result contiguous

* adding zero frequency to distributed sht

* small hotfix for lobatto grid precomputation routine

* adding lobatto grid to tests

* adding caching

* replacing many numpy calls with torch calls

* bumping up version number to 0.7.6

* Improved computation of Morlet filter basis and switched to a Hann window.

* addresses #064 and some cleanup

Without putting `signal` on `device`, we get `RuntimeError: Expected all tensors to be on the same device` when `sht` is called.

switched psi tensor computation to double precision and implemented a fudge factor for theta_cutoff to avoid aliasing issues with the grid width