Unverified Commit bde9e227 authored by Tao He's avatar Tao He Committed by GitHub
Browse files

[Bugfix][Qwen] fixes the weights dtype in qwen3_next: it is actually a bfloat16 (#27030)


Signed-off-by: default avatarTao He <linzhu.ht@alibaba-inc.com>
parent 08405609
......@@ -325,7 +325,6 @@ class Qwen3NextGatedDeltaNet(nn.Module, MambaBase):
self.A_log = nn.Parameter(
torch.empty(
divide(self.num_v_heads, self.tp_size),
dtype=torch.float32,
)
)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment