transformer_engine/pytorch/transformer.py · 156a074a979efef9220763537d324217816691e0 · OpenDAS / TransformerEngine

[PyTorch] Rotary Position Embedding (#246) · 156a074a

Kirthi Shankar Sivamani authored May 25, 2023



* Rotary Position Embedding
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* remove einops
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* Improve docstr
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

---------
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

156a074a

transformer.py 24.3 KB

Replace transformer.py