Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
6eabc6cb
Unverified
Commit
6eabc6cb
authored
Jun 27, 2024
by
Cyrus Leung
Committed by
GitHub
Jun 26, 2024
Browse files
[Doc] Add note about context length in Phi-3-Vision example (#5887)
parent
2110557d
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
1 deletion
+3
-1
examples/phi3v_example.py
examples/phi3v_example.py
+3
-1
No files found.
examples/phi3v_example.py
View file @
6eabc6cb
...
@@ -9,6 +9,9 @@ from vllm.multimodal.image import ImagePixelData
...
@@ -9,6 +9,9 @@ from vllm.multimodal.image import ImagePixelData
def
run_phi3v
():
def
run_phi3v
():
model_path
=
"microsoft/Phi-3-vision-128k-instruct"
model_path
=
"microsoft/Phi-3-vision-128k-instruct"
# Note: The model has 128k context length by default which may cause OOM
# If that's the case, override `max_model_len` with a smaller value via args
llm
=
LLM
(
llm
=
LLM
(
model
=
model_path
,
model
=
model_path
,
trust_remote_code
=
True
,
trust_remote_code
=
True
,
...
@@ -16,7 +19,6 @@ def run_phi3v():
...
@@ -16,7 +19,6 @@ def run_phi3v():
image_token_id
=
32044
,
image_token_id
=
32044
,
image_input_shape
=
"1,3,1008,1344"
,
image_input_shape
=
"1,3,1008,1344"
,
image_feature_size
=
1921
,
image_feature_size
=
1921
,
disable_image_processor
=
False
,
)
)
image
=
Image
.
open
(
"images/cherry_blossom.jpg"
)
image
=
Image
.
open
(
"images/cherry_blossom.jpg"
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment