"...git@developer.sourcefind.cn:modelzoo/tsm_pytorch.git" did not exist on "5b3e36dc8846dc35232928c57cc979a82d58ecf5"
Unverified Commit b48c7f7b authored by Arthur's avatar Arthur Committed by GitHub
Browse files

[T5 doc] Fix confusing documentation about `d_kv` (#21896)

* Confusing documentation in T5

* Fix onfusing documentation in T5 configuration file
parent edbb37f7
...@@ -48,8 +48,8 @@ class T5Config(PretrainedConfig): ...@@ -48,8 +48,8 @@ class T5Config(PretrainedConfig):
d_model (`int`, *optional*, defaults to 512): d_model (`int`, *optional*, defaults to 512):
Size of the encoder layers and the pooler layer. Size of the encoder layers and the pooler layer.
d_kv (`int`, *optional*, defaults to 64): d_kv (`int`, *optional*, defaults to 64):
Size of the key, query, value projections per attention head. `d_kv` has to be equal to `d_model // Size of the key, query, value projections per attention head. The `inner_dim` of the projection layer will
num_heads`. be defined as `num_heads * d_kv`.
d_ff (`int`, *optional*, defaults to 2048): d_ff (`int`, *optional*, defaults to 2048):
Size of the intermediate feed forward layer in each `T5Block`. Size of the intermediate feed forward layer in each `T5Block`.
num_layers (`int`, *optional*, defaults to 6): num_layers (`int`, *optional*, defaults to 6):
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment