Unverified Commit 886fcbdd authored by strgrb's avatar strgrb Committed by GitHub
Browse files

Use apply_rope_with_cos_sin_cache_inplace for DeepSeek (#4764)


Co-authored-by: default avatarZhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
parent 8bf6d7f4
...@@ -645,7 +645,7 @@ class DeepseekScalingRotaryEmbedding(RotaryEmbedding): ...@@ -645,7 +645,7 @@ class DeepseekScalingRotaryEmbedding(RotaryEmbedding):
cache = torch.cat((cos, sin), dim=-1) cache = torch.cat((cos, sin), dim=-1)
return cache return cache
def forward( def forward_native(
self, self,
positions: torch.Tensor, positions: torch.Tensor,
query: torch.Tensor, query: torch.Tensor,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment