transformer_engine/pytorch/transformer.py · 5a881a0834384460c2558cfbc40bc9ff0a6aa146 · OpenDAS / TransformerEngine · GitLab

Find file Blame History Permalink

Catch FA internal error with compute capability 8.6 (#113) · 5a881a08
Kirthi Shankar Sivamani authored Mar 22, 2023
```
FA doesn't support compute 8.6 with head_dim>64
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
```
5a881a08

transformer.py 49.9 KB