Unverified Commit 4df69506 authored by Sylvain Gugger's avatar Sylvain Gugger Committed by GitHub
Browse files

Fix YosoConfig doc (#15353)

parent fc8fc400
...@@ -67,17 +67,17 @@ class YosoConfig(PretrainedConfig): ...@@ -67,17 +67,17 @@ class YosoConfig(PretrainedConfig):
The epsilon used by the layer normalization layers. The epsilon used by the layer normalization layers.
position_embedding_type (`str`, *optional*, defaults to `"absolute"`): position_embedding_type (`str`, *optional*, defaults to `"absolute"`):
Type of position embedding. Choose one of `"absolute"`, `"relative_key"`, `"relative_key_query"`. Type of position embedding. Choose one of `"absolute"`, `"relative_key"`, `"relative_key_query"`.
use_expectation (*bool*, *optional*, defaults to *True*): use_expectation (`bool`, *optional*, defaults to `True`):
Whether or not to use YOSO Expectation. Overrides any effect of num_hash. Whether or not to use YOSO Expectation. Overrides any effect of num_hash.
hash_code_len (`int`, *optional*, defaults to 9): hash_code_len (`int`, *optional*, defaults to 9):
The length of hashes generated by the hash functions. The length of hashes generated by the hash functions.
num_hash (`int`, *optional*, defaults to 64): num_hash (`int`, *optional*, defaults to 64):
Number of hash functions used in [`YosoSelfAttention`]. Number of hash functions used in [`YosoSelfAttention`].
conv_window (`int`, *optional*, defaults to None): conv_window (`int`, *optional*):
Kernel size of depth-wise convolution. Kernel size of depth-wise convolution.
use_fast_hash (*bool*, *optional*, defaults to *False*): use_fast_hash (`bool`, *optional*, defaults to `False`):
Whether or not to use custom cuda kernels which perform fast random projection via hadamard transform. Whether or not to use custom cuda kernels which perform fast random projection via hadamard transform.
lsh_backward (*bool*, *optional*, defaults to *True*): lsh_backward (`bool`, *optional*, defaults to `True`):
Whether or not to perform backpropagation using Locality Sensitive Hashing. Whether or not to perform backpropagation using Locality Sensitive Hashing.
Example: Example:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment