Unverified Commit 54ab804e authored by Hojin Yang's avatar Hojin Yang Committed by GitHub
Browse files

[Bugfix] Store Qwen3Next A_log in fp32 (#37810)


Signed-off-by: default avatareffortprogrammer <yhjhoward7@gmail.com>
Co-authored-by: default avatarRoger Wang <hey@rogerw.io>
parent 02e6efe5
...@@ -501,6 +501,7 @@ class Qwen3NextGatedDeltaNet(nn.Module, MambaBase): ...@@ -501,6 +501,7 @@ class Qwen3NextGatedDeltaNet(nn.Module, MambaBase):
self.A_log = nn.Parameter( self.A_log = nn.Parameter(
torch.empty( torch.empty(
divide(self.num_v_heads, self.tp_size), divide(self.num_v_heads, self.tp_size),
dtype=torch.float32,
) )
) )
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment