Commit 11135bbc authored by zhuwenwen's avatar zhuwenwen
Browse files

update readme

parent 66dadd2d
......@@ -113,7 +113,7 @@ python -m vllm.entrypoints.api_server --model meta-llama/Llama-2-7b-chat-hf --
```bash
python benchmarks/benchmark_serving.py --model meta-llama/Llama-2-7b-chat-hf --dataset ShareGPT_V3_unfiltered_cleaned_split.json --num-prompts 1 --trust-remote-code
```
参数同使用数据集,离线批量推理性能测试,具体参考[benchmarks/benchmark_serving.py]
参数同使用数据集,离线批量推理性能测试,具体参考[benchmarks/benchmark_serving.py](benchmarks/benchmark_serving.py)
### OpenAI兼容服务
......@@ -139,7 +139,7 @@ curl http://localhost:8000/v1/completions \
"temperature": 0
}'
```
或者使用[examples/openai_completion_client.py]
或者使用[examples/openai_completion_client.py](examples/openai_completion_client.py)
### OpenAI Chat API和vllm结合使用
......@@ -154,7 +154,7 @@ curl http://localhost:8000/v1/chat/completions \
]
}'
```
或者使用[examples/openai_chatcompletion_client.py]
或者使用[examples/openai_chatcompletion_client.py](examples/openai_chatcompletion_client.py)
## result
使用的加速卡:1张 DCU-K100_AI-64G
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment