"torchvision/vscode:/vscode.git/clone" did not exist on "3bfdb4275c13ec8207ae78222b7895c01a01371b"
Commit 85b50c88 authored by Chen Chen's avatar Chen Chen Committed by A. Unique TensorFlower
Browse files

Add TalkingHeadsAttention to README.

PiperOrigin-RevId: 307082084
parent f499e880
......@@ -14,6 +14,9 @@ If `from_tensor` and `to_tensor` are the same, then this is self-attention.
* [CachedAttention](attention.py) implements an attention layer with cache used
for auto-agressive decoding.
* [TalkingHeadsAttention](talking_heads_attention.py) implements the talking
heads attention, as decribed in ["Talking-Heads Attention"](https://arxiv.org/abs/2003.02436).
* [Transformer](transformer.py) implements an optionally masked transformer as
described in ["Attention Is All You Need"](https://arxiv.org/abs/1706.03762).
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment