Commit 37759753 authored by raojy's avatar raojy
Browse files

updata

parent 01b0f097
......@@ -35,23 +35,6 @@
- 挂载地址`-v`根据实际模型情况修改
```bash
docker run -it \
--shm-size 60g \
--network=host \
--name {docker_name} \
--privileged \
--device=/dev/kfd \
--device=/dev/dri \
--device=/dev/mkfd \
--group-add video \
--cap-add=SYS_PTRACE \
--security-opt seccomp=unconfined \
-u root \
-v /opt/hyhal/:/opt/hyhal/:ro \
-v /path/your_code_data/:/path/your_code_data/ \
{docker_image_name} bash
示例如下:
docker run -it \
--shm-size 60g \
--network=host \
......@@ -271,8 +254,7 @@ curl http://localhost:8000/v1/chat/completions \
# 适用于72B模型
# 启动命令
vllm serve "/home/project/weight_cache/models--Qwen--Qwen2.5-VL-72B-Instruct/models--Qwen--Qwen2.5-VL-72B-Instruct/snapshots/89c86200743eec961a297729e7990e8f2ddbc4c5" \
vllm serve Qwen/Qwen2.5-VL-72B-Instruct \
--served-model-name "qwen-vl" \
--tensor-parallel-size 4 \
--gpu-memory-utilization 0.95 \
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment