GitLab

[DCU]Fix some bugs

See merge request dcutoolkit/deeplearing/TransformerEngine!56

Signed-off-by: zhaochao <zhaochao1@sugon.com>

Signed-off-by: zhaochao <zhaochao1@sugon.com>

Signed-off-by: zhaochao <zhaochao1@sugon.com>

Signed-off-by: zhaochao <zhaochao1@sugon.com>

Signed-off-by: zhaochao <zhaochao1@sugon.com>

Signed-off-by: zhaochao <zhaochao1@sugon.com>

Signed-off-by: zhaochao <zhaochao1@sugon.com>

[DCU]Fix memory overflow and test-didistributed in L1_pytorch_istributed_unittest

See merge request dcutoolkit/deeplearing/TransformerEngine!49

Signed-off-by: Tangao <2205747538@qq.com>

Merge branch 'develop_v2.8' of http://10.16.6.30/dcutoolkit/deeplearing/TransformerEngine into release_v2.8

Update activation offload code to align with the official version

See merge request dcutoolkit/deeplearing/TransformerEngine!52

Merge branch 'develop_v2.8' of http://10.16.6.30/dcutoolkit/deeplearing/TransformerEngine into release_v2.8

Merge branch 'develop_v2.8' of http://10.16.6.30/dcutoolkit/deeplearing/TransformerEngine into develop_v2.8

Signed-off-by: Tangao <2205747538@qq.com>

Merge branch 'release_v2.8' of http://10.16.6.30/dcutoolkit/deeplearing/TransformerEngine into release_v2.8

Signed-off-by: wenjh <wenjh@sugon.com>

Merge branch 'develop_v2.8' of http://10.16.6.30/dcutoolkit/deeplearing/TransformerEngine into release_v2.8

Merge branch 'develop_v2.8' of http://10.16.6.30/dcutoolkit/deeplearing/TransformerEngine into release_v2.8

Signed-off-by: wenjh <wenjh@sugon.com>

Signed-off-by: wenjh <wenjh@sugon.com>

support activation offloading

See merge request dcutoolkit/deeplearing/TransformerEngine!44

* Make sure to set usages for linear op quantizers before forward
Signed-off-by: Tim Moon <tmoon@nvidia.com>

* Avoid unsupported case for fused dbias+quantize kernel

Hopper does not support dbias + FP8 cast without FP8 transpose.
Signed-off-by: Tim Moon <tmoon@nvidia.com>

---------
Signed-off-by: Tim Moon <tmoon@nvidia.com>

* Fix the cublas workspace alignment
Signed-off-by: Przemek Tredak <ptredak@nvidia.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci



* Fix
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Przemyslaw Tredak <ptrendx@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci



---------
Signed-off-by: Przemek Tredak <ptredak@nvidia.com>
Signed-off-by: Przemyslaw Tredak <ptrendx@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Load modules during initialize
Signed-off-by: Jeremy Berchtold <jberchtold@nvidia.com>
Co-authored-by: JAX Toolbox <jax@nvidia.com>