docs/examples/advanced_optimizations.ipynb · 9ff2c07655d11d3fb9d88d84a0d9032b58108e33 · OpenDAS / TransformerEngine

"...AutoBuildImmortalWrt.git" did not exist on "52c83aca70dccc72f9205759a9c2e338d4aaee76"

Use correct FP8 group in multi-GPU docs (#852) · 9ff2c076

Tim Moon authored May 28, 2024



* Use correct FP8 group in multi-GPU docs

FP8 process group should be tensor-parallel group
Signed-off-by: Tim Moon <tmoon@nvidia.com>

* Synchronize FP8 scales over world group in multi-GPU docs
Signed-off-by: Tim Moon <tmoon@nvidia.com>

---------
Signed-off-by: Tim Moon <tmoon@nvidia.com>

9ff2c076

advanced_optimizations.ipynb 13.7 KB

Replace advanced_optimizations.ipynb