README.md 308 Bytes
Newer Older
Tri Dao's avatar
Tri Dao committed
1
This CUDA extension implements fused dropout + residual + LayerNorm, based on
2
3
4
5
6
Apex's [FastLayerNorm](https://github.com/NVIDIA/apex/tree/master/apex/contrib/layer_norm).
We add dropout and residual, and make it work for both pre-norm and post-norm architecture.
```sh
cd csrc/layer_norm && pip install .
```