Skip to content

GitLab

Unverified Commit c65863ce authored by Raza Habib's avatar Raza Habib Committed by GitHub
Browse files

Remove duplicated mish activation function (#7856) 

* Remove duplicated mish activation function

* Update activations.py
parent f5c45a19
Hide whitespace changes
Inline Side-by-side
Showing with 0 additions and 4 deletions
src/transformers/activations.py
View file @ c65863ce
...@@ -66,7 +66,3 @@ def get_activation(activation_string): ...@@ -66,7 +66,3 @@ def get_activation(activation_string):
return ACT2FN[activation_string] return ACT2FN[activation_string]
else: else:
raise KeyError("function {} not found in ACT2FN mapping {}".format(activation_string, list(ACT2FN.keys()))) raise KeyError("function {} not found in ACT2FN mapping {}".format(activation_string, list(ACT2FN.keys())))
def mish(x):
return x * torch.tanh(torch.nn.functional.softplus(x))
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment