tests/paddle/test_operators.py · 2da34d4101905054890cb6bc0d3a2a7e468bb4ae · OpenDAS / TransformerEngine

Change scaling factor from E8M0 to E8M23 (#427) · 2da34d41

Kirthi Shankar Sivamani authored Sep 22, 2023



* Change scaling factor from E8M0 to E8M23
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* fix formula
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

---------
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

2da34d41

test_operators.py 34.4 KB

Replace test_operators.py