Commit 2d237d09 authored by chenych's avatar chenych
Browse files

update readme

parent 8a519462
......@@ -19,7 +19,7 @@ Qwen3嵌入模型系列是Qwen3家族最新的专有模型,专门为文本嵌
### Docker(方法一)
```bash
docker pull image.sourcefind.cn:5000/dcu/admin/base/custom:vllm0.8.5-ubuntu22.04-dtk25.04-rc7-das1.5-py3.10-20250612-fixpy-rocblas0611-rc2
docker pull docker pull image.sourcefind.cn:5000/dcu/admin/base/vllm:0.8.5-ubuntu22.04-dtk25.04.1-rc5-das1.6-py3.10-20250711
docker run -it --shm-size 200g --network=host --name {docker_name} --privileged --device=/dev/kfd --device=/dev/dri --device=/dev/mkfd --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -u root -v /path/your_code_data/:/path/your_code_data/ -v /opt/hyhal/:/opt/hyhal/:ro {imageID} bash
cd /your_code_path/qwen3-reranker_pytorch
......@@ -44,7 +44,6 @@ python: 3.10
vllm: 0.8.5
torch: 2.4.1+das.opt2.dtk2504
deepspeed: 0.14.2+das.opt2.dtk2504
```
`Tips:以上dtk驱动、python、torch等DCU相关工具版本需要严格一一对应`
......@@ -61,6 +60,9 @@ pip install transformers>=4.51.0
## 推理
### vllm推理方法
vllm 0.8.5不支持serve模式启动推理,offline方式请参考项目脚本`infer_vllm.py`
#### offline
```bash
## 必须添加HF_ENDPOINT环境变量
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment