Commit 0ac03a9b authored by zhuwenwen's avatar zhuwenwen
Browse files

update readme

parent 47286aad
......@@ -116,7 +116,7 @@ python benchmarks/benchmark_throughput.py --num-prompts 1 --model THUDM/glm-4-9b
### api服务推理性能测试
1、启动服务端:
```bash
python -m vllm.entrypoints.api_server --model THUDM/glm-4-9b-chat --dtype float16 --enforce-eager -tp 1
python -m vllm.entrypoints.openai.api_server --model THUDM/glm-4-9b-chat --dtype float16 --enforce-eager -tp 1
```
2、启动客户端:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment