transformer_engine/paddle/fp8.py · c32a62ccb0a38d14d98e8bd6a14ace2d73bbac25 · OpenDAS / TransformerEngine

[Paddle] Eliminate amax update bubbles by using custom_ops (#436) · c32a62cc

zlsh80826 authored Sep 22, 2023



* Eliminate amax_and_scale_update bubbles
Signed-off-by: rewang <rewang@nvidia.com>

* Add CUDA check
Signed-off-by: rewang <rewang@nvidia.com>

---------
Signed-off-by: rewang <rewang@nvidia.com>

c32a62cc

fp8.py 9.65 KB

Replace fp8.py