Commit 85b50c88 authored by Chen Chen's avatar Chen Chen Committed by A. Unique TensorFlower
Browse files

Add TalkingHeadsAttention to README.

PiperOrigin-RevId: 307082084
parent f499e880
...@@ -14,6 +14,9 @@ If `from_tensor` and `to_tensor` are the same, then this is self-attention. ...@@ -14,6 +14,9 @@ If `from_tensor` and `to_tensor` are the same, then this is self-attention.
* [CachedAttention](attention.py) implements an attention layer with cache used * [CachedAttention](attention.py) implements an attention layer with cache used
for auto-agressive decoding. for auto-agressive decoding.
* [TalkingHeadsAttention](talking_heads_attention.py) implements the talking
heads attention, as decribed in ["Talking-Heads Attention"](https://arxiv.org/abs/2003.02436).
* [Transformer](transformer.py) implements an optionally masked transformer as * [Transformer](transformer.py) implements an optionally masked transformer as
described in ["Attention Is All You Need"](https://arxiv.org/abs/1706.03762). described in ["Attention Is All You Need"](https://arxiv.org/abs/1706.03762).
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment