Skip to content

GitLab

"git@developer.sourcefind.cn:chenpangpang/transformers.git" did not exist on "b230181d413aeb39cc4edd7bf9d905eb61733416"
Unverified Commit 8c4471d1 authored by Sylvain Gugger's avatar Sylvain Gugger Committed by GitHub
Browse files

Fix ESM models buffers (#24576) 

* Fix ESM models buffers

* Remove modifs

* Tied weights keys are needed silly

* quality
parent b324557a
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
src/transformers/models/esm/modeling_esm.py
View file @ 8c4471d1
...@@ -96,7 +96,7 @@ class RotaryEmbedding(torch.nn.Module): ...@@ -96,7 +96,7 @@ class RotaryEmbedding(torch.nn.Module):
# Generate and save the inverse frequency buffer (non trainable) # Generate and save the inverse frequency buffer (non trainable)
inv_freq = 1.0 / (10000 ** (torch.arange(0, dim, 2).float() / dim)) inv_freq = 1.0 / (10000 ** (torch.arange(0, dim, 2).float() / dim))
inv_freq = inv_freq inv_freq = inv_freq
self.register_buffer("inv_freq", inv_freq, persistent=False) self.register_buffer("inv_freq", inv_freq)
self._seq_len_cached = None self._seq_len_cached = None
self._cos_cached = None self._cos_cached = None
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment