Unverified Commit 4cf94298 authored by Wentao Ye's avatar Wentao Ye Committed by GitHub
Browse files

[Bug] Fix `error 'Dynamo failed to run FX node with fake tensors` for Deepseek V3.2 (#31046)


Signed-off-by: default avataryewentao256 <zhyanwentao@126.com>
parent 83a317f6
......@@ -878,8 +878,11 @@ class Indexer(nn.Module):
)
q_pe, k_pe = rotary_emb(positions, q_pe, k_pe.unsqueeze(1))
q = torch.cat([q_pe.squeeze(0), q_nope], dim=-1)
k = torch.cat([k_pe.squeeze((0, 2)), k_nope], dim=-1)
# `rotary_emb` is shape-preserving; `q_pe` is already
# [num_tokens, n_head, rope_dim].
q = torch.cat([q_pe, q_nope], dim=-1)
# `k_pe` is [num_tokens, 1, rope_dim] (MQA).
k = torch.cat([k_pe.squeeze(1), k_nope], dim=-1)
# we only quant q here since k quant is fused with cache insertion
q = q.view(-1, self.head_dim)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment