Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
329df38f
Unverified
Commit
329df38f
authored
Jun 28, 2024
by
Roger Wang
Committed by
GitHub
Jun 29, 2024
Browse files
[Misc] Update Phi-3-Vision Example (#5981)
Co-authored-by:
Cyrus Leung
<
cyrus.tl.leung@gmail.com
>
parent
580353da
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
3 deletions
+5
-3
examples/phi3v_example.py
examples/phi3v_example.py
+5
-3
No files found.
examples/phi3v_example.py
View file @
329df38f
...
@@ -10,8 +10,10 @@ from vllm.multimodal.image import ImagePixelData
...
@@ -10,8 +10,10 @@ from vllm.multimodal.image import ImagePixelData
def
run_phi3v
():
def
run_phi3v
():
model_path
=
"microsoft/Phi-3-vision-128k-instruct"
model_path
=
"microsoft/Phi-3-vision-128k-instruct"
# Note: The model has 128k context length by default which may cause OOM
# Note: The default setting of max_num_seqs (256) and
# In this example, we override max_model_len to 2048.
# max_model_len (128k) for this model may cause OOM.
# In this example, we override max_num_seqs to 5 while
# keeping the original context length of 128k.
llm
=
LLM
(
llm
=
LLM
(
model
=
model_path
,
model
=
model_path
,
trust_remote_code
=
True
,
trust_remote_code
=
True
,
...
@@ -19,7 +21,7 @@ def run_phi3v():
...
@@ -19,7 +21,7 @@ def run_phi3v():
image_token_id
=
32044
,
image_token_id
=
32044
,
image_input_shape
=
"1,3,1008,1344"
,
image_input_shape
=
"1,3,1008,1344"
,
image_feature_size
=
1921
,
image_feature_size
=
1921
,
max_
model_len
=
2048
,
max_
num_seqs
=
5
,
)
)
image
=
Image
.
open
(
"images/cherry_blossom.jpg"
)
image
=
Image
.
open
(
"images/cherry_blossom.jpg"
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment