GitLab

All integer parameters are specialized by default, so the two parameters
removed in this commit could lead to kernel re-compilation, even if
they were completely unused.

* Enable paged attention in varlen forward

* Format + fix padding

Optimize compile to 1: avoid oom 2: minimize swap usage 3: avoid threads starvation when letting ninja decide how many workers to spawn or manual MAX_JOBS "guesses". Logic is to take the min value of MAX_JOBS auto-calculated by two metrics: 1: cpu cores 2: free memory. This should allow flash-attn to compile close to the most efficient manner under any consumer/server env. (#832)

For faster and smaller builds in some simple cases,
provide switches to allow disabling
-backward
-alibi
-uneven k
-dropout
-local attention
Co-authored-by: Jeremy Francis Reizenstein <bottler@users.noreply.github.com>

* [CI] Build wheels for Pytorch 2.3 (dev/nightly)

Resolves #790
Signed-off-by: Christian Kadner <ckadner@us.ibm.com>

* update TORCH_CUDA_VERSION
Signed-off-by: Christian Kadner <ckadner@us.ibm.com>

* revert torch 2.2 back to dev20231130
Signed-off-by: Christian Kadner <ckadner@us.ibm.com>

* add link to PyTorch compatibility matrix
Signed-off-by: Christian Kadner <ckadner@us.ibm.com>

---------
Signed-off-by: Christian Kadner <ckadner@us.ibm.com>

* Updated docstrings of bert_padding.py

Added docstrings for missing arguments in the unpad and pad methods.

* Update bert_padding.py

Fixed spelling mistakes

Signed-off-by: Tao He <sighingnow@gmail.com>

Co-authored-by: ljss <450993438@qq.com>