Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
ModelZoo
Qwen3.5
Commits
162c2b41
Commit
162c2b41
authored
Apr 09, 2026
by
chenych
Browse files
Update packages version.
parent
92b6a63b
Changes
3
Show whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
13 additions
and
11 deletions
+13
-11
README.md
README.md
+13
-11
doc/result-dcu.jpg
doc/result-dcu.jpg
+0
-0
doc/result-dcu.png
doc/result-dcu.png
+0
-0
No files found.
README.md
View file @
162c2b41
...
@@ -16,10 +16,10 @@ Qwen3.5 通过异构基础设施实现高效的原生多模态训练:在视觉
...
@@ -16,10 +16,10 @@ Qwen3.5 通过异构基础设施实现高效的原生多模态训练:在视觉
| :------: |:-----------------------------------------:|
| :------: |:-----------------------------------------:|
| DTK | 26.04 |
| DTK | 26.04 |
| python | 3.10.12 |
| python | 3.10.12 |
| transformers | 5.2.0
.dev0
|
| transformers | 5.2.0 |
| vllm |
0.15.1+das.opt1.alpha.dtk2604
|
| vllm | 0.15.1+das.opt1.alpha.dtk2604
.torch290.2604042155.gba9f96
|
| triton | 3.3.0+das.opt2.dtk2604.
2026
0203.g3
93ad86c
|
| triton | 3.3.0+das.opt2.dtk2604.
torch290.2
02
6
03
31
.g3
1542e
|
| torch | 2.9.0+das.opt1.dtk2604.20260
126.g22910426
|
| torch | 2.9.0+das.opt1.dtk2604.20260
331.g4e3c1e7
|
当前仅支持定制镜像: harbor.sourcefind.cn:5443/dcu/admin/base/custom:vllm015-ubuntu22.04-dtk26.04-glm5-0408
当前仅支持定制镜像: harbor.sourcefind.cn:5443/dcu/admin/base/custom:vllm015-ubuntu22.04-dtk26.04-glm5-0408
...
@@ -52,8 +52,8 @@ docker run -it \
...
@@ -52,8 +52,8 @@ docker run -it \
## 推理
## 推理
### vllm
### vllm
**注意**
:
**注意**
:
-
使用
`K100 AI`
启动服务时需要添加
`--disable-custom-all-reduce`
参数
-
使用
`K100 AI`
型号的卡,
启动服务时需要添加
`--disable-custom-all-reduce`
参数
-
加载W8A8模型启动服务时需要添加
`-cc.mode=3`
和
`-cc.inductor_compile_config='{"combo_kernels": false, "benchmark_combo_kernel": false}'`
参数
-
加载W8A8
量化
模型
,
启动服务时需要添加
`-cc.mode=3`
和
`-cc.inductor_compile_config='{"combo_kernels": false, "benchmark_combo_kernel": false}'`
参数
#### 单机推理
#### 单机推理
```
bash
```
bash
...
@@ -88,7 +88,8 @@ curl http://localhost:8001/v1/chat/completions \
...
@@ -88,7 +88,8 @@ curl http://localhost:8001/v1/chat/completions \
"messages": [
"messages": [
{"role": "user", "content": "Type \"I love Qwen3.5\" backwards"}
{"role": "user", "content": "Type \"I love Qwen3.5\" backwards"}
],
],
"temperature": 0.6
"temperature": 0.6,
"max_tokens": 512
}'
}'
```
```
...
@@ -175,13 +176,14 @@ curl http://localhost:8001/v1/chat/completions \
...
@@ -175,13 +176,14 @@ curl http://localhost:8001/v1/chat/completions \
"messages": [
"messages": [
{"role": "user", "content": "Type \"I love Qwen3.5\" backwards"}
{"role": "user", "content": "Type \"I love Qwen3.5\" backwards"}
],
],
"temperature": 0.6
"temperature": 0.6,
"max_tokens": 512
}'
}'
```
```
## 效果展示
## 效果展示
<div
align=
center
>
<div
align=
center
>
<img
src=
"./doc/result-dcu.
j
pg"
/>
<img
src=
"./doc/result-dcu.p
n
g"
/>
</div>
</div>
### 精度
### 精度
...
...
doc/result-dcu.jpg
deleted
100644 → 0
View file @
92b6a63b
519 KB
doc/result-dcu.png
0 → 100644
View file @
162c2b41
146 KB
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment