Commit 1070a611 authored by zhuwenwen's avatar zhuwenwen
Browse files

update readme

parent 2603e417
...@@ -107,7 +107,7 @@ python benchmarks/benchmark_throughput.py --num-prompts 1 --model meta-llama/Lla ...@@ -107,7 +107,7 @@ python benchmarks/benchmark_throughput.py --num-prompts 1 --model meta-llama/Lla
### api服务推理性能测试 ### api服务推理性能测试
1、启动服务端: 1、启动服务端:
```bash ```bash
python -m vllm.entrypoints.api_server --model meta-llama/Llama-2-7b-chat-hf --dtype float16 --enforce-eager -tp 1 python -m vllm.entrypoints.openai.api_server --model meta-llama/Llama-2-7b-chat-hf --dtype float16 --enforce-eager -tp 1
``` ```
2、启动客户端: 2、启动客户端:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment