Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
b7d30986
Unverified
Commit
b7d30986
authored
Feb 16, 2025
by
Roger Wang
Committed by
GitHub
Feb 16, 2025
Browse files
[V1] Update doc and examples for H2O-VL (#13349)
Signed-off-by:
Roger Wang
<
ywang@roblox.com
>
parent
dc0f7ccf
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
6 additions
and
7 deletions
+6
-7
docs/source/models/supported_models.md
docs/source/models/supported_models.md
+2
-2
examples/offline_inference/vision_language.py
examples/offline_inference/vision_language.py
+2
-2
examples/offline_inference/vision_language_multi_image.py
examples/offline_inference/vision_language_multi_image.py
+2
-3
No files found.
docs/source/models/supported_models.md
View file @
b7d30986
...
@@ -726,7 +726,7 @@ See [this page](#generative-models) for more information on how to use generativ
...
@@ -726,7 +726,7 @@ See [this page](#generative-models) for more information on how to use generativ
*
`h2oai/h2ovl-mississippi-800m`
,
`h2oai/h2ovl-mississippi-2b`
, etc.
*
`h2oai/h2ovl-mississippi-800m`
,
`h2oai/h2ovl-mississippi-2b`
, etc.
*
*
*
✅︎
*
✅︎
*
\*
*
✅︎
\*
-
*
`Idefics3ForConditionalGeneration`
-
*
`Idefics3ForConditionalGeneration`
*
Idefics3
*
Idefics3
*
T + I
*
T + I
...
@@ -869,7 +869,7 @@ See [this page](#generative-models) for more information on how to use generativ
...
@@ -869,7 +869,7 @@ See [this page](#generative-models) for more information on how to use generativ
<sup>
+
</sup>
Multiple items can be inputted per text prompt for this modality.
<sup>
+
</sup>
Multiple items can be inputted per text prompt for this modality.
:::{note}
:::{note}
H2O-VL series models
will be available in V1 once we support backends other than FlashAttention.
`h2oai/h2ovl-mississippi-2b`
will be available in V1 once we support backends other than FlashAttention.
:::
:::
:::{note}
:::{note}
...
...
examples/offline_inference/vision_language.py
View file @
b7d30986
...
@@ -119,7 +119,7 @@ def run_glm4v(question: str, modality: str):
...
@@ -119,7 +119,7 @@ def run_glm4v(question: str, modality: str):
def
run_h2ovl
(
question
:
str
,
modality
:
str
):
def
run_h2ovl
(
question
:
str
,
modality
:
str
):
assert
modality
==
"image"
assert
modality
==
"image"
model_name
=
"h2oai/h2ovl-mississippi-
2b
"
model_name
=
"h2oai/h2ovl-mississippi-
800m
"
llm
=
LLM
(
llm
=
LLM
(
model
=
model_name
,
model
=
model_name
,
...
@@ -136,7 +136,7 @@ def run_h2ovl(question: str, modality: str):
...
@@ -136,7 +136,7 @@ def run_h2ovl(question: str, modality: str):
add_generation_prompt
=
True
)
add_generation_prompt
=
True
)
# Stop tokens for H2OVL-Mississippi
# Stop tokens for H2OVL-Mississippi
# https://huggingface.co/h2oai/h2ovl-mississippi-
2b
# https://huggingface.co/h2oai/h2ovl-mississippi-
800m
stop_token_ids
=
[
tokenizer
.
eos_token_id
]
stop_token_ids
=
[
tokenizer
.
eos_token_id
]
return
llm
,
prompt
,
stop_token_ids
return
llm
,
prompt
,
stop_token_ids
...
...
examples/offline_inference/vision_language_multi_image.py
View file @
b7d30986
...
@@ -78,14 +78,13 @@ def load_deepseek_vl2(question: str, image_urls: List[str]):
...
@@ -78,14 +78,13 @@ def load_deepseek_vl2(question: str, image_urls: List[str]):
def
load_h2ovl
(
question
:
str
,
image_urls
:
List
[
str
])
->
ModelRequestData
:
def
load_h2ovl
(
question
:
str
,
image_urls
:
List
[
str
])
->
ModelRequestData
:
model_name
=
"h2oai/h2ovl-mississippi-
2b
"
model_name
=
"h2oai/h2ovl-mississippi-
800m
"
llm
=
LLM
(
llm
=
LLM
(
model
=
model_name
,
model
=
model_name
,
trust_remote_code
=
True
,
trust_remote_code
=
True
,
max_model_len
=
8192
,
max_model_len
=
8192
,
limit_mm_per_prompt
=
{
"image"
:
len
(
image_urls
)},
limit_mm_per_prompt
=
{
"image"
:
len
(
image_urls
)},
mm_processor_kwargs
=
{
"max_dynamic_patch"
:
4
},
)
)
placeholders
=
"
\n
"
.
join
(
f
"Image-
{
i
}
: <image>
\n
"
placeholders
=
"
\n
"
.
join
(
f
"Image-
{
i
}
: <image>
\n
"
...
@@ -99,7 +98,7 @@ def load_h2ovl(question: str, image_urls: List[str]) -> ModelRequestData:
...
@@ -99,7 +98,7 @@ def load_h2ovl(question: str, image_urls: List[str]) -> ModelRequestData:
add_generation_prompt
=
True
)
add_generation_prompt
=
True
)
# Stop tokens for H2OVL-Mississippi
# Stop tokens for H2OVL-Mississippi
# https://huggingface.co/h2oai/h2ovl-mississippi-
2b
# https://huggingface.co/h2oai/h2ovl-mississippi-
800m
stop_token_ids
=
[
tokenizer
.
eos_token_id
]
stop_token_ids
=
[
tokenizer
.
eos_token_id
]
return
ModelRequestData
(
return
ModelRequestData
(
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment