Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
laibao
llava_vllm
Commits
3a876679
Commit
3a876679
authored
Oct 16, 2024
by
laibao
Browse files
No commit message
No commit message
parent
091528b0
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
15 additions
and
14 deletions
+15
-14
README.md
README.md
+15
-14
No files found.
README.md
View file @
3a876679
...
...
@@ -124,29 +124,30 @@ python examples/llava_example.py
启动服务:
```
bash
python
-m
vllm.entrypoints.openai.api_server
--model
/
llava/llava-1.5-7b-hf
--image-input-type
pixel_values
--image-token-id
32000
--image-input-shape
1,3,336,336
--image-feature-size
576
--chat-template
template_llava.jinja
python
-m
vllm.entrypoints.openai.api_server
--model
llava/llava-1.5-7b-hf
--image-input-type
pixel_values
--image-token-id
32000
--image-input-shape
1,3,336,336
--image-feature-size
576
--chat-template
template_llava.jinja
```
这里
`--model`
为加载模型路径,
`--image-input-type pixel_values`
为图片输入的类型:pixel_values,
`--image-token-id`
用于指定图片输入的特殊标记 ID,
`--image-input-shape`
设置图片输入的形状,
`--image-feature-size`
指定图像特征的大小,
`--chat-template`
可以添加新模板覆盖默认模板。
列出模型型号:
### OpenAI Completions API和vllm结合使用
```
bash
curl http://localhost:8000/v1/models
```
### OpenAI Completions API和vllm结合使用
curl http://localhost:8000/v1/chat/completions
\
-H
"Content-Type: application/json"
\
-H
"Authorization: Bearer EMPTY"
\
-d
'{
"model": "llava/llava-1.5-7b-hf",
"messages": [
{
"role": "user",
"content": "What is the content of this image? [local file](images/cherry_blossom.jpg)"
}
],
"max_tokens": 300
}'
```
bash
curl http://localhost:8000/v1/completions
\
-H
"Content-Type: application/json"
\
-d
'{
"model": "Qwen/Qwen1.5-7B",
"prompt": "What is deep learning?",
"max_tokens": 7,
"temperature": 0
}'
```
或者使用
[
examples/openai_completion_client.py
](
examples/openai_completion_client.py
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment