Skip to content

GitLab

Commit bec01fd9 authored by zhuwenwen's avatar zhuwenwen
Browse files

add vllm tp set

parent c1e81762
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
README.md
View file @ bec01fd9
...@@ -73,7 +73,7 @@ python tools/list_configs.py llama mmlu ...@@ -73,7 +73,7 @@ python tools/list_configs.py llama mmlu
```shell ```shell
python run.py configs/vllm/eval_llama2_vllm.py python run.py configs/vllm/eval_llama2_vllm.py
``` ```
其它模型使用参考`configs/vllm/eval_xxx_vllm.py` 其它模型使用参考`configs/vllm/eval_xxx_vllm.py`,若使用多卡，修改`tensor_parallel_size`和`num_gpus`为卡的数量
2、使用lmdeploy推理验证 2、使用lmdeploy推理验证
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment