"docs/vscode:/vscode.git/clone" did not exist on "110df74332785ee749af47c5a3eb634d216b8f3b"
Unverified Commit bd02164c authored by Michael Goin's avatar Michael Goin Committed by GitHub
Browse files

Bugfix for whisper quantization due to fake k_proj bias (#12524)


Signed-off-by: default avatarmgoin <michael@neuralmagic.com>
parent 46fb0567
...@@ -743,7 +743,7 @@ def _create_fake_bias_for_k_proj( ...@@ -743,7 +743,7 @@ def _create_fake_bias_for_k_proj(
So that the bias for k_proj in qkv_proj can be initialized with zeros. So that the bias for k_proj in qkv_proj can be initialized with zeros.
""" """
for name, weight in weights: for name, weight in weights:
if ".self_attn.k_proj.weight" in name: if name.endswith(".self_attn.k_proj.weight"):
bias = torch.zeros(weight.size(0)) bias = torch.zeros(weight.size(0))
bias_name = name.replace("weight", "bias") bias_name = name.replace("weight", "bias")
yield from [(name, weight), (bias_name, bias)] yield from [(name, weight), (bias_name, bias)]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment