Unverified Commit d017bceb authored by Roger Young's avatar Roger Young Committed by GitHub
Browse files

[BugFix] Fix minimax m2 model rotary_dim (#30384)


Signed-off-by: default avatarxuebi <xuebi@minimaxi.com>
Co-authored-by: default avatarxuebi <xuebi@minimaxi.com>
parent cebda2a4
......@@ -201,7 +201,7 @@ class MiniMaxM2Attention(nn.Module):
self.rotary_emb = get_rope(
self.head_dim,
rotary_dim=rotary_dim,
rotary_dim=self.head_dim,
max_position=max_position_embeddings,
rope_parameters=rope_parameters,
)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment