transformer_engine/pytorch/csrc/extensions.h · 35bbe740e834b7491d6304cb39b8c003ac7abb5c · OpenDAS / TransformerEngine

[PyTorch] Let Fused RoPE support CP with THD format (#1238) · 55dcbb4b

Xin Yao authored Oct 12, 2024



* Let Fused RoPE support THD with CP
Signed-off-by: Xin Yao <xiny@nvidia.com>

* add comment
Signed-off-by: Xin Yao <xiny@nvidia.com>

---------
Signed-off-by: Xin Yao <xiny@nvidia.com>
Co-authored-by: Xiaowei Ren <103958965+xrennvidia@users.noreply.github.com>

55dcbb4b

extensions.h 31.2 KB

Replace extensions.h