Unverified Commit d3cf61b8 authored by Qiming Zhang's avatar Qiming Zhang Committed by GitHub
Browse files

fix gemma3 results all zero (#17364)


Signed-off-by: default avatarmayuyuace <qiming1.zhang@intel.com>
parent a39203f9
......@@ -241,6 +241,9 @@ class GemmaRMSNorm(CustomOp):
"""PyTorch-native implementation equivalent to forward()."""
orig_dtype = x.dtype
if residual is not None:
if orig_dtype == torch.float16:
x = x + residual.float()
else:
x = x + residual
residual = x
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment