Unverified Commit e531cd2f authored by Isaac Ong's avatar Isaac Ong Committed by GitHub
Browse files

[PyTorch] Fix MultiheadAttention docstring (#634)



Fix MHA docstring
Signed-off-by: default avatarIsaac Ong <isaacong.jw@gmail.com>
parent 6c1a8bb5
...@@ -2909,8 +2909,8 @@ class MultiheadAttention(torch.nn.Module): ...@@ -2909,8 +2909,8 @@ class MultiheadAttention(torch.nn.Module):
together with the output of the linear transformation. together with the output of the linear transformation.
Example use case: residual connection for transformer module is Example use case: residual connection for transformer module is
taken post layernorm. taken post layernorm.
input_layernorm: bool, default = `True` input_layernorm: bool, default = `False`
if set to `False`, layer normalization to the input is not applied. if set to `True`, layer normalization to the input is applied.
attention_type: { 'self', 'cross' }, default = 'self' attention_type: { 'self', 'cross' }, default = 'self'
type of attention applied. type of attention applied.
zero_centered_gamma : bool, default = 'False' zero_centered_gamma : bool, default = 'False'
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment