Unverified Commit 886fcbdd authored by strgrb's avatar strgrb Committed by GitHub
Browse files

Use apply_rope_with_cos_sin_cache_inplace for DeepSeek (#4764)


Co-authored-by: default avatarZhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
parent 8bf6d7f4
......@@ -645,7 +645,7 @@ class DeepseekScalingRotaryEmbedding(RotaryEmbedding):
cache = torch.cat((cos, sin), dim=-1)
return cache
def forward(
def forward_native(
self,
positions: torch.Tensor,
query: torch.Tensor,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment