Commit 3a876679 authored by laibao's avatar laibao
Browse files

No commit message

No commit message
parent 091528b0
...@@ -124,29 +124,30 @@ python examples/llava_example.py ...@@ -124,29 +124,30 @@ python examples/llava_example.py
启动服务: 启动服务:
```bash ```bash
python -m vllm.entrypoints.openai.api_server --model /llava/llava-1.5-7b-hf --image-input-type pixel_values --image-token-id 32000 --image-input-shape 1,3,336,336 --image-feature-size 576 --chat-template template_llava.jinja python -m vllm.entrypoints.openai.api_server --model llava/llava-1.5-7b-hf --image-input-type pixel_values --image-token-id 32000 --image-input-shape 1,3,336,336 --image-feature-size 576 --chat-template template_llava.jinja
``` ```
这里 `--model`为加载模型路径,`--image-input-type pixel_values`为图片输入的类型:pixel_values,`--image-token-id`用于指定图片输入的特殊标记 ID,`--image-input-shape`设置图片输入的形状,`--image-feature-size`指定图像特征的大小,`--chat-template`可以添加新模板覆盖默认模板。 这里 `--model`为加载模型路径,`--image-input-type pixel_values`为图片输入的类型:pixel_values,`--image-token-id`用于指定图片输入的特殊标记 ID,`--image-input-shape`设置图片输入的形状,`--image-feature-size`指定图像特征的大小,`--chat-template`可以添加新模板覆盖默认模板。
列出模型型号:
```bash
curl http://localhost:8000/v1/models
```
### OpenAI Completions API和vllm结合使用 ### OpenAI Completions API和vllm结合使用
```bash ```bash
curl http://localhost:8000/v1/completions \
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-H "Authorization: Bearer EMPTY" \
-d '{ -d '{
"model": "Qwen/Qwen1.5-7B", "model": "llava/llava-1.5-7b-hf",
"prompt": "What is deep learning?", "messages": [
"max_tokens": 7, {
"temperature": 0 "role": "user",
"content": "What is the content of this image? [local file](images/cherry_blossom.jpg)"
}
],
"max_tokens": 300
}' }'
``` ```
或者使用[examples/openai_completion_client.py](examples/openai_completion_client.py) 或者使用[examples/openai_completion_client.py](examples/openai_completion_client.py)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment