transformer_engine/paddle/csrc/custom_ops.cu · c32a62ccb0a38d14d98e8bd6a14ace2d73bbac25 · OpenDAS / TransformerEngine

[Paddle] Eliminate amax update bubbles by using custom_ops (#436) · c32a62cc

zlsh80826 authored Sep 22, 2023



* Eliminate amax_and_scale_update bubbles
Signed-off-by: rewang <rewang@nvidia.com>

* Add CUDA check
Signed-off-by: rewang <rewang@nvidia.com>

---------
Signed-off-by: rewang <rewang@nvidia.com>

c32a62cc

custom_ops.cu 63.5 KB

Replace custom_ops.cu