Commits · f351191b38f3da1c9dcd4b8cfe0bba7c59d63f5d · OpenDAS / TransformerEngine

14 Jul, 2023 1 commit
- [PyTorch] Fix FP8 checkpointing for non forward execution cases (#323) · f351191b
  Kirthi Shankar Sivamani authored Jul 14, 2023
```
Bug fix for checkpointing
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
```
  f351191b
02 Jun, 2023 1 commit

Fix some Pylance errors (#259) · 144e4888

Jan Bielak authored Jun 02, 2023



* Ignore IDE files
Signed-off-by: Jan Bielak <jbielak@nvidia.com>

* Fix typing errors
Signed-off-by: Jan Bielak <jbielak@nvidia.com>

* Ignore devcontainer files
Signed-off-by: Jan Bielak <jbielak@nvidia.com>

* Avoid import from private module
Signed-off-by: Jan Bielak <jbielak@nvidia.com>

* Apply @timmoon10 's suggestions
Signed-off-by: Jan Bielak <jbielak@nvidia.com>

---------
Signed-off-by: Jan Bielak <jbielak@nvidia.com>

144e4888

01 Jun, 2023 1 commit

Don't save fp8 weight tensors if `is_first_microbatch` is None (#244) · 80825fde

Sudhakar Singh authored Jun 01, 2023



* extend fp8 weight placeholders logic for Linear, LNLinear, LNMLP
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>

* Update transformer_engine/pytorch/module/base.py
Co-authored-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>

* Update transformer_engine/pytorch/module/base.py
Co-authored-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>

* Update transformer_engine/pytorch/module/base.py
Co-authored-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>

* Update transformer_engine/pytorch/module/base.py
Co-authored-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>

* Update transformer_engine/pytorch/module/base.py
Co-authored-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>

* Update transformer_engine/pytorch/module/layernorm_linear.py
Co-authored-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>

* Update transformer_engine/pytorch/module/layernorm_mlp.py
Co-authored-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>

* Update transformer_engine/pytorch/module/linear.py
Co-authored-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>

* Update linear.py
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>

* Update layernorm_linear.py
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>

* Update layernorm_mlp.py
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>

* lint
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

---------
Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
Co-authored-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

80825fde

25 May, 2023 1 commit

Clearer error messages for dtype and shape assertions (#245) · 871fdf51

Carlos Mocholí authored May 24, 2023



* Clearer error messages for dtype and shape assertions
Signed-off-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Update transformer_engine/pytorch/utils.py
Co-authored-by: Tim Moon <4406448+timmoon10@users.noreply.github.com>
Signed-off-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Update transformer_engine/pytorch/utils.py
Co-authored-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
Signed-off-by: Carlos Mocholí <carlossmocholi@gmail.com>

---------
Signed-off-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Tim Moon <4406448+timmoon10@users.noreply.github.com>
Co-authored-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

871fdf51

09 May, 2023 1 commit

PyTorch refactor (#201) · c6a4a4e0

Kirthi Shankar Sivamani authored May 09, 2023



* Initial refactor
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* refactor attention out of transformer.py
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* Fix ONNX export
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

* linting
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

---------
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>

c6a4a4e0