Skip to content

GitLab

Unverified Commit 47ef7946 authored by Tolga Cangöz's avatar Tolga Cangöz Committed by GitHub
Browse files

Apply Occam's Razor in position embedding calculation (#11562) 

* fix: remove redundant indexing

* style
parent b272807b
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 3 deletions
src/diffusers/models/embeddings.py
View file @ 47ef7946
...@@ -1149,9 +1149,7 @@ def get_1d_rotary_pos_embed( ...@@ -1149,9 +1149,7 @@ def get_1d_rotary_pos_embed(
theta = theta * ntk_factor theta = theta * ntk_factor
freqs = ( freqs = (
1.0 1.0 / (theta ** (torch.arange(0, dim, 2, dtype=freqs_dtype, device=pos.device) / dim)) / linear_factor
/ (theta ** (torch.arange(0, dim, 2, dtype=freqs_dtype, device=pos.device)[: (dim // 2)] / dim))
/ linear_factor
) # [D/2] ) # [D/2]
freqs = torch.outer(pos, freqs) # type: ignore # [S, D/2] freqs = torch.outer(pos, freqs) # type: ignore # [S, D/2]
is_npu = freqs.device.type == "npu" is_npu = freqs.device.type == "npu"
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment