Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
Qwen3.5_vllm
Commits
162c2b41
Commit
162c2b41
authored
Apr 09, 2026
by
chenych
Browse files
Update packages version.
parent
92b6a63b
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
13 additions
and
11 deletions
+13
-11
README.md
README.md
+13
-11
doc/result-dcu.jpg
doc/result-dcu.jpg
+0
-0
doc/result-dcu.png
doc/result-dcu.png
+0
-0
No files found.
README.md
View file @
162c2b41
...
@@ -14,12 +14,12 @@ Qwen3.5 通过异构基础设施实现高效的原生多模态训练:在视觉
...
@@ -14,12 +14,12 @@ Qwen3.5 通过异构基础设施实现高效的原生多模态训练:在视觉
## 环境依赖
## 环境依赖
| 软件 | 版本 |
| 软件 | 版本 |
| :------: |:-----------------------------------------:|
| :------: |:-----------------------------------------:|
| DTK | 26.04
|
| DTK | 26.04 |
| python | 3.10.12
|
| python | 3.10.12 |
| transformers | 5.2.0
.dev0
|
| transformers | 5.2.0 |
| vllm |
0.15.1+das.opt1.alpha.dtk2604
|
| vllm | 0.15.1+das.opt1.alpha.dtk2604
.torch290.2604042155.gba9f96
|
| triton | 3.3.0+das.opt2.dtk2604.
2026
0203.g3
93ad86c
|
| triton | 3.3.0+das.opt2.dtk2604.
torch290.2
02
6
03
31
.g3
1542e
|
| torch | 2.9.0+das.opt1.dtk2604.20260
126.g22910426
|
| torch | 2.9.0+das.opt1.dtk2604.20260
331.g4e3c1e7
|
当前仅支持定制镜像: harbor.sourcefind.cn:5443/dcu/admin/base/custom:vllm015-ubuntu22.04-dtk26.04-glm5-0408
当前仅支持定制镜像: harbor.sourcefind.cn:5443/dcu/admin/base/custom:vllm015-ubuntu22.04-dtk26.04-glm5-0408
...
@@ -52,8 +52,8 @@ docker run -it \
...
@@ -52,8 +52,8 @@ docker run -it \
## 推理
## 推理
### vllm
### vllm
**注意**
:
**注意**
:
-
使用
`K100 AI`
启动服务时需要添加
`--disable-custom-all-reduce`
参数
-
使用
`K100 AI`
型号的卡,
启动服务时需要添加
`--disable-custom-all-reduce`
参数
-
加载W8A8模型启动服务时需要添加
`-cc.mode=3`
和
`-cc.inductor_compile_config='{"combo_kernels": false, "benchmark_combo_kernel": false}'`
参数
-
加载W8A8
量化
模型
,
启动服务时需要添加
`-cc.mode=3`
和
`-cc.inductor_compile_config='{"combo_kernels": false, "benchmark_combo_kernel": false}'`
参数
#### 单机推理
#### 单机推理
```
bash
```
bash
...
@@ -88,7 +88,8 @@ curl http://localhost:8001/v1/chat/completions \
...
@@ -88,7 +88,8 @@ curl http://localhost:8001/v1/chat/completions \
"messages": [
"messages": [
{"role": "user", "content": "Type \"I love Qwen3.5\" backwards"}
{"role": "user", "content": "Type \"I love Qwen3.5\" backwards"}
],
],
"temperature": 0.6
"temperature": 0.6,
"max_tokens": 512
}'
}'
```
```
...
@@ -175,13 +176,14 @@ curl http://localhost:8001/v1/chat/completions \
...
@@ -175,13 +176,14 @@ curl http://localhost:8001/v1/chat/completions \
"messages": [
"messages": [
{"role": "user", "content": "Type \"I love Qwen3.5\" backwards"}
{"role": "user", "content": "Type \"I love Qwen3.5\" backwards"}
],
],
"temperature": 0.6
"temperature": 0.6,
"max_tokens": 512
}'
}'
```
```
## 效果展示
## 效果展示
<div
align=
center
>
<div
align=
center
>
<img
src=
"./doc/result-dcu.
j
pg"
/>
<img
src=
"./doc/result-dcu.p
n
g"
/>
</div>
</div>
### 精度
### 精度
...
...
doc/result-dcu.jpg
deleted
100644 → 0
View file @
92b6a63b
519 KB
doc/result-dcu.png
0 → 100644
View file @
162c2b41
146 KB
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment