"git@developer.sourcefind.cn:wangsen/mineru.git" did not exist on "6323f28b250158579e5d77a0f1025f9841885f4a"
Fix stuff related to the causal_mask in CodeGen. (#21527)
* Fix stuff related to the causal_mask in CodeGen. 1. Line 613, `_keys_to_ignore_on_load_missing = [r"h\.\d+\.attn\.masked_bias", r"h\.\d+\.attn\.bias"]` => `_keys_to_ignore_on_load_missing = [r"h\.\d+\.attn\.causal_mask"]` to load correctly from CodeGen checkpoint without `causal_mask`. 2. Line 152, `causal_mask = self.causal_mask[:, :, key_length - query_length : key_length, :key_length] ` => `causal_mask = self.causal_mask[:, :, key_length - query_length : key_length, :key_length].bool() ` to alleviate potential user warning saying like `UserWarning: where received a uint8 condition tensor. This behavior is deprecated and will be removed in a future version of PyTorch. Use a boolean condition instead.`. * Revert the .bool() Revert the .bool() and leave it to the future PR.
Showing
Please register or sign in to comment