Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
Qwen3-VL-Reranker_vllm
Commits
a72e268a
Commit
a72e268a
authored
Mar 10, 2026
by
weishb
Browse files
support vllm_0.15.1
parent
bd12e34d
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
8 additions
and
11 deletions
+8
-11
README.md
README.md
+8
-11
vllm.zip
vllm.zip
+0
-0
No files found.
README.md
View file @
a72e268a
...
...
@@ -19,9 +19,9 @@ Qwen3-VL-Embedding 和 Qwen3-VL-Reranker 模型系列是通义千问(Qwen)
| DTK | 26.04 |
| python | 3.10.12 |
| transformers | 4.57.6 |
| vllm | 0.1
1.0
+das.opt1.
rc2
.dtk2604 |
| vllm | 0.1
5.1
+das.opt1.
alpha
.dtk2604 |
推荐使用镜像:harbor.sourcefind.cn:5443/dcu/admin/base/vllm
:
0.1
1.0
-ubuntu22.04-dtk26.04-0130-py3.10-202602
0
2
推荐使用镜像:harbor.sourcefind.cn:5443/dcu/admin/base/
custom:
vllm0.1
5.1
-ubuntu22.04-dtk26.04-0130-py3.10-2026022
0
```
bash
docker run
-it
\
...
...
@@ -38,7 +38,7 @@ docker run -it \
-u
root
\
-v
/opt/hyhal/:/opt/hyhal/:ro
\
-v
/path/your_code_data/:/path/your_code_data/
\
harbor.sourcefind.cn:5443/dcu/admin/base/vllm
:
0.1
1.0
-ubuntu22.04-dtk26.04-0130-py3.10-202602
0
2 bash
harbor.sourcefind.cn:5443/dcu/admin/base/
custom:
vllm0.1
5.1
-ubuntu22.04-dtk26.04-0130-py3.10-2026022
0
bash
```
更多镜像可前往
[
光源
](
https://sourcefind.cn/#/service-list
)
下载使用。
...
...
@@ -46,9 +46,7 @@ docker run -it \
镜像内其他环境配置
```
1.解压vllm.zip到/usr/local/lib/python3.10/dist-packages直接覆盖需要修改的文件
unzip -o vllm.zip -d /usr/local/lib/python3.10/dist-packages
pip install pycountry
```
...
...
@@ -62,18 +60,17 @@ docker run -it \
### vllm
#### 单机推理
启动服务
```
bash
```
bash
## serve启动
vllm serve Qwen/Qwen3-VL-Reranker-8B
\
--runner
pooling
\
--hf-overrides
'{"architectures": ["Qwen3VLForSequenceClassification"],"classifier_from_token":["no","yes"],"is_original_qwen3_reranker":true}'
\
--chat-template
qwen3_vl_reranker.jinja
\
--max-model-len
4096
\
--served-model-name
qwen3-vl-reranker
```
调用服务:
```
## client访问
curl
-s
http://127.0.0.1:8000/rerank
\
-H
"Content-Type: application/json"
\
-d
'{
...
...
vllm.zip
deleted
100644 → 0
View file @
bd12e34d
File suppressed by a .gitattributes entry or the file's encoding is unsupported.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment