transformer_engine/common/activation/activation_template.h · d99142a0177a2462cbda07a31aaa8e68b4e85461 · OpenDAS / TransformerEngine

[JAX] SwiGLU Implementation (#773) · f85553ea

Phuong Nguyen authored Apr 24, 2024



* Implemented swiglu and silu
Signed-off-by: Phuong Nguyen <phuonguyen@nvidia.com>

* Renamed nvte-*silu to nvte-*swish + generalized GetDBiasDact functions
Signed-off-by: Phuong Nguyen <phuonguyen@nvidia.com>

f85553ea

activation_template.h 5.55 KB

Replace activation_template.h