Unverified Commit b7d30986 authored by Roger Wang's avatar Roger Wang Committed by GitHub
Browse files

[V1] Update doc and examples for H2O-VL (#13349)


Signed-off-by: default avatarRoger Wang <ywang@roblox.com>
parent dc0f7ccf
...@@ -726,7 +726,7 @@ See [this page](#generative-models) for more information on how to use generativ ...@@ -726,7 +726,7 @@ See [this page](#generative-models) for more information on how to use generativ
* `h2oai/h2ovl-mississippi-800m`, `h2oai/h2ovl-mississippi-2b`, etc. * `h2oai/h2ovl-mississippi-800m`, `h2oai/h2ovl-mississippi-2b`, etc.
* *
* ✅︎ * ✅︎
* \* * ✅︎\*
- * `Idefics3ForConditionalGeneration` - * `Idefics3ForConditionalGeneration`
* Idefics3 * Idefics3
* T + I * T + I
...@@ -869,7 +869,7 @@ See [this page](#generative-models) for more information on how to use generativ ...@@ -869,7 +869,7 @@ See [this page](#generative-models) for more information on how to use generativ
<sup>+</sup> Multiple items can be inputted per text prompt for this modality. <sup>+</sup> Multiple items can be inputted per text prompt for this modality.
:::{note} :::{note}
H2O-VL series models will be available in V1 once we support backends other than FlashAttention. `h2oai/h2ovl-mississippi-2b` will be available in V1 once we support backends other than FlashAttention.
::: :::
:::{note} :::{note}
......
...@@ -119,7 +119,7 @@ def run_glm4v(question: str, modality: str): ...@@ -119,7 +119,7 @@ def run_glm4v(question: str, modality: str):
def run_h2ovl(question: str, modality: str): def run_h2ovl(question: str, modality: str):
assert modality == "image" assert modality == "image"
model_name = "h2oai/h2ovl-mississippi-2b" model_name = "h2oai/h2ovl-mississippi-800m"
llm = LLM( llm = LLM(
model=model_name, model=model_name,
...@@ -136,7 +136,7 @@ def run_h2ovl(question: str, modality: str): ...@@ -136,7 +136,7 @@ def run_h2ovl(question: str, modality: str):
add_generation_prompt=True) add_generation_prompt=True)
# Stop tokens for H2OVL-Mississippi # Stop tokens for H2OVL-Mississippi
# https://huggingface.co/h2oai/h2ovl-mississippi-2b # https://huggingface.co/h2oai/h2ovl-mississippi-800m
stop_token_ids = [tokenizer.eos_token_id] stop_token_ids = [tokenizer.eos_token_id]
return llm, prompt, stop_token_ids return llm, prompt, stop_token_ids
......
...@@ -78,14 +78,13 @@ def load_deepseek_vl2(question: str, image_urls: List[str]): ...@@ -78,14 +78,13 @@ def load_deepseek_vl2(question: str, image_urls: List[str]):
def load_h2ovl(question: str, image_urls: List[str]) -> ModelRequestData: def load_h2ovl(question: str, image_urls: List[str]) -> ModelRequestData:
model_name = "h2oai/h2ovl-mississippi-2b" model_name = "h2oai/h2ovl-mississippi-800m"
llm = LLM( llm = LLM(
model=model_name, model=model_name,
trust_remote_code=True, trust_remote_code=True,
max_model_len=8192, max_model_len=8192,
limit_mm_per_prompt={"image": len(image_urls)}, limit_mm_per_prompt={"image": len(image_urls)},
mm_processor_kwargs={"max_dynamic_patch": 4},
) )
placeholders = "\n".join(f"Image-{i}: <image>\n" placeholders = "\n".join(f"Image-{i}: <image>\n"
...@@ -99,7 +98,7 @@ def load_h2ovl(question: str, image_urls: List[str]) -> ModelRequestData: ...@@ -99,7 +98,7 @@ def load_h2ovl(question: str, image_urls: List[str]) -> ModelRequestData:
add_generation_prompt=True) add_generation_prompt=True)
# Stop tokens for H2OVL-Mississippi # Stop tokens for H2OVL-Mississippi
# https://huggingface.co/h2oai/h2ovl-mississippi-2b # https://huggingface.co/h2oai/h2ovl-mississippi-800m
stop_token_ids = [tokenizer.eos_token_id] stop_token_ids = [tokenizer.eos_token_id]
return ModelRequestData( return ModelRequestData(
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment