Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
Qwen3-Reranker_pytorch
Commits
2d237d09
Commit
2d237d09
authored
Jul 21, 2025
by
chenych
Browse files
update readme
parent
8a519462
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
2 deletions
+4
-2
README.md
README.md
+4
-2
No files found.
README.md
View file @
2d237d09
...
...
@@ -19,7 +19,7 @@ Qwen3嵌入模型系列是Qwen3家族最新的专有模型,专门为文本嵌
### Docker(方法一)
```
bash
docker pull image.sourcefind.cn:5000/dcu/admin/base/
custom:
vllm0.8.5-ubuntu22.04-dtk25.04-rc
7
-das1.
5
-py3.10-20250
612-fixpy-rocblas0611-rc2
docker pull
docker pull image.sourcefind.cn:5000/dcu/admin/base/vllm
:
0.8.5-ubuntu22.04-dtk25.04
.1
-rc
5
-das1.
6
-py3.10-20250
711
docker run
-it
--shm-size
200g
--network
=
host
--name
{
docker_name
}
--privileged
--device
=
/dev/kfd
--device
=
/dev/dri
--device
=
/dev/mkfd
--group-add
video
--cap-add
=
SYS_PTRACE
--security-opt
seccomp
=
unconfined
-u
root
-v
/path/your_code_data/:/path/your_code_data/
-v
/opt/hyhal/:/opt/hyhal/:ro
{
imageID
}
bash
cd
/your_code_path/qwen3-reranker_pytorch
...
...
@@ -44,7 +44,6 @@ python: 3.10
vllm: 0.8.5
torch: 2.4.1+das.opt2.dtk2504
deepspeed: 0.14.2+das.opt2.dtk2504
```
`Tips:以上dtk驱动、python、torch等DCU相关工具版本需要严格一一对应`
...
...
@@ -61,6 +60,9 @@ pip install transformers>=4.51.0
## 推理
### vllm推理方法
vllm 0.8.5不支持serve模式启动推理,offline方式请参考项目脚本
`infer_vllm.py`
。
#### offline
```
bash
## 必须添加HF_ENDPOINT环境变量
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment