Commit 3a876679 authored by laibao's avatar laibao
Browse files

No commit message

No commit message
parent 091528b0
......@@ -124,29 +124,30 @@ python examples/llava_example.py
启动服务:
```bash
python -m vllm.entrypoints.openai.api_server --model /llava/llava-1.5-7b-hf --image-input-type pixel_values --image-token-id 32000 --image-input-shape 1,3,336,336 --image-feature-size 576 --chat-template template_llava.jinja
python -m vllm.entrypoints.openai.api_server --model llava/llava-1.5-7b-hf --image-input-type pixel_values --image-token-id 32000 --image-input-shape 1,3,336,336 --image-feature-size 576 --chat-template template_llava.jinja
```
这里 `--model`为加载模型路径,`--image-input-type pixel_values`为图片输入的类型:pixel_values,`--image-token-id`用于指定图片输入的特殊标记 ID,`--image-input-shape`设置图片输入的形状,`--image-feature-size`指定图像特征的大小,`--chat-template`可以添加新模板覆盖默认模板。
列出模型型号:
### OpenAI Completions API和vllm结合使用
```bash
curl http://localhost:8000/v1/models
```
### OpenAI Completions API和vllm结合使用
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer EMPTY" \
-d '{
"model": "llava/llava-1.5-7b-hf",
"messages": [
{
"role": "user",
"content": "What is the content of this image? [local file](images/cherry_blossom.jpg)"
}
],
"max_tokens": 300
}'
```bash
curl http://localhost:8000/v1/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Qwen/Qwen1.5-7B",
"prompt": "What is deep learning?",
"max_tokens": 7,
"temperature": 0
}'
```
或者使用[examples/openai_completion_client.py](examples/openai_completion_client.py)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment