Unverified Commit 9e910c46 authored by Teriks's avatar Teriks Committed by GitHub
Browse files

Fix SD2.X clip single file load projection_dim (#10770)



* Fix SD2.X clip single file load projection_dim

Infer projection_dim from the checkpoint before loading
from pretrained, override any incorrect hub config.

Hub configuration for SD2.X specifies projection_dim=512
which is incorrect for SD2.X checkpoints loaded from civitai
and similar.

Exception was previously thrown upon attempting to
load_model_dict_into_meta for SD2.X single file checkpoints.

Such LDM models usually require projection_dim=1024

* convert_open_clip_checkpoint use hidden_size for text_proj_dim

* convert_open_clip_checkpoint, revert checkpoint[text_proj_key].shape[1] -> [0]

values are identical

---------
Co-authored-by: default avatarTeriks <Teriks@users.noreply.github.com>
Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
parent 5e3b7d2d
...@@ -1448,8 +1448,8 @@ def convert_open_clip_checkpoint( ...@@ -1448,8 +1448,8 @@ def convert_open_clip_checkpoint(
if text_proj_key in checkpoint: if text_proj_key in checkpoint:
text_proj_dim = int(checkpoint[text_proj_key].shape[0]) text_proj_dim = int(checkpoint[text_proj_key].shape[0])
elif hasattr(text_model.config, "projection_dim"): elif hasattr(text_model.config, "hidden_size"):
text_proj_dim = text_model.config.projection_dim text_proj_dim = text_model.config.hidden_size
else: else:
text_proj_dim = LDM_OPEN_CLIP_TEXT_PROJECTION_DIM text_proj_dim = LDM_OPEN_CLIP_TEXT_PROJECTION_DIM
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment