transformer_engine/pytorch/attention.py · 74eb7c33cc0667503d81be4af6d493a64d8871d1 · OpenDAS / TransformerEngine

Xiaowei Ren authored Nov 03, 2023



fix bwd error with FA v2
Signed-off-by: Xiaowei Ren <xren@nvidia.com>
Co-authored-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

74eb7c33

attention.py 130 KB