"vscode:/vscode.git/clone" did not exist on "b58c5f28aa687de58e0d314eb1d43a3e26359689"
Unverified Commit b1cc4dfe authored by Isotr0py's avatar Isotr0py Committed by GitHub
Browse files

[VLM] Support loading InternVideo2.5 models as original InternVLChatModel (#14738)


Signed-off-by: default avatarIsotr0py <2037008807@qq.com>
parent 38240392
......@@ -786,9 +786,9 @@ See [this page](#generative-models) for more information on how to use generativ
*
* ✅︎
- * `InternVLChatModel`
* InternVL 2.5, Mono-InternVL, InternVL 2.0
* InternVideo 2.5, InternVL 2.5, Mono-InternVL, InternVL 2.0
* T + I<sup>E+</sup>
* `OpenGVLab/InternVL2_5-4B`, `OpenGVLab/Mono-InternVL-2B`, `OpenGVLab/InternVL2-4B`, etc.
* `OpenGVLab/InternVideo2_5_Chat_8B`, `OpenGVLab/InternVL2_5-4B`, `OpenGVLab/Mono-InternVL-2B`, `OpenGVLab/InternVL2-4B`, etc.
*
* ✅︎
* ✅︎
......
......@@ -981,5 +981,12 @@ class InternVLChatModel(nn.Module, SupportsMultiModal, SupportsPP):
def load_weights(self, weights: Iterable[Tuple[str,
torch.Tensor]]) -> Set[str]:
loader = AutoWeightsLoader(self)
# unused modules appear in OpenGVLab/InternVideo2_5_Chat_8B
skip_prefixes = [
"action_embed", "temporal_embed", "track_embed",
"track_embed_decoder", "box_token", "cg_criterion", "cg_model",
"loc_encoder", "loc_decoder", "sam", "temporal_token",
"track_token"
]
loader = AutoWeightsLoader(self, skip_prefixes=skip_prefixes)
return loader.load_weights(weights)
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment