Commit f499e880 authored by Le Hou's avatar Le Hou Committed by A. Unique TensorFlower
Browse files

Add ReZeroTransformer to README.

PiperOrigin-RevId: 307079383
parent 3c9198bc
......@@ -17,6 +17,9 @@ for auto-agressive decoding.
* [Transformer](transformer.py) implements an optionally masked transformer as
described in ["Attention Is All You Need"](https://arxiv.org/abs/1706.03762).
* [ReZeroTransformer](rezero_transformer.py) implements Transformer with ReZero
described in ["ReZero is All You Need: Fast Convergence at Large Depth"](https://arxiv.org/abs/2003.04887).
* [OnDeviceEmbedding](on_device_embedding.py) implements efficient embedding lookups designed for TPU-based models.
* [PositionalEmbedding](position_embedding.py) creates a positional embedding
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment