GitLab

WIP because there seems to be some race conditions for head dimensions other
than 16, 32, 64, 128.

build wheel and upload to release

Previously we were using block_size=256.

They don't have to have the same block size, number of threads, etc.

fix typo in function mha_fwd

as title.

build wheel workflow

remove numpy dependency

According to the `setup.py` file, only dependencies are torch and einops. But the `bert_padding.py` file requires `numpy` only to multiply the elements of a `torch.Size` object. This change aims at allowing the use of FlashAttention without numpy.

Make flash attention compile on Windows.