transformer_engine/pytorch/transformer.py · 2b05e121a469301db0d3e9d6299f8c723ee2b6c1 · OpenDAS / TransformerEngine

"cacheflow/model_executor/parallel_utils/parallel_state.py" did not exist on "21b3671bbc508662561ae95a418a26dbe71db356"

[PyTorch] Support L2Normalization basic op -> use for qk_norm (#1864) · 40a30a5f

Evgeny Tsykunov authored Jun 12, 2025



* Support L2Norm basic op
Signed-off-by: Evgeny <etsykunov@nvidia.com>

* Add L2Norm module wrapper
Signed-off-by: Evgeny <etsykunov@nvidia.com>

* Expose qk_norm to MHA nd transformer laayer
Signed-off-by: Evgeny <etsykunov@nvidia.com>

* Move tests into separate file
Signed-off-by: Evgeny <etsykunov@nvidia.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci



* fix pass
Signed-off-by: Evgeny <etsykunov@nvidia.com>

* Add license
Signed-off-by: Evgeny <etsykunov@nvidia.com>

* Remove  module
Signed-off-by: Evgeny <etsykunov@nvidia.com>

* Resollve comments
Signed-off-by: Evgeny <etsykunov@nvidia.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci



---------
Signed-off-by: Evgeny <etsykunov@nvidia.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

40a30a5f

transformer.py 41.1 KB

Replace transformer.py