tests/pytorch/test_onnx_export.py · 81429b80388b3e0b3b7a746ed3568694a2fdd5eb · OpenDAS / TransformerEngine

deprecate qk layer scaling and fp32 softmax args (#90) · 81429b80

Kirthi Shankar Sivamani authored Mar 11, 2023



* deprecate qk layer scaling and fp32 softmax args
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* apply QK layer scaling for fp16 training
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* address review comments
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

---------
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

81429b80

test_onnx_export.py 35.4 KB

Replace test_onnx_export.py