Commit 162c2b41 authored by chenych's avatar chenych
Browse files

Update packages version.

parent 92b6a63b
...@@ -16,10 +16,10 @@ Qwen3.5 通过异构基础设施实现高效的原生多模态训练:在视觉 ...@@ -16,10 +16,10 @@ Qwen3.5 通过异构基础设施实现高效的原生多模态训练:在视觉
| :------: |:-----------------------------------------:| | :------: |:-----------------------------------------:|
| DTK | 26.04 | | DTK | 26.04 |
| python | 3.10.12 | | python | 3.10.12 |
| transformers | 5.2.0.dev0 | | transformers | 5.2.0 |
| vllm | 0.15.1+das.opt1.alpha.dtk2604 | | vllm | 0.15.1+das.opt1.alpha.dtk2604.torch290.2604042155.gba9f96 |
| triton | 3.3.0+das.opt2.dtk2604.20260203.g393ad86c | | triton | 3.3.0+das.opt2.dtk2604.torch290.20260331.g31542e |
| torch | 2.9.0+das.opt1.dtk2604.20260126.g22910426 | | torch | 2.9.0+das.opt1.dtk2604.20260331.g4e3c1e7 |
当前仅支持定制镜像: harbor.sourcefind.cn:5443/dcu/admin/base/custom:vllm015-ubuntu22.04-dtk26.04-glm5-0408 当前仅支持定制镜像: harbor.sourcefind.cn:5443/dcu/admin/base/custom:vllm015-ubuntu22.04-dtk26.04-glm5-0408
...@@ -52,8 +52,8 @@ docker run -it \ ...@@ -52,8 +52,8 @@ docker run -it \
## 推理 ## 推理
### vllm ### vllm
**注意**: **注意**:
- 使用`K100 AI` 启动服务时需要添加`--disable-custom-all-reduce`参数 - 使用`K100 AI`型号的卡,启动服务时需要添加`--disable-custom-all-reduce`参数
- 加载W8A8模型启动服务时需要添加`-cc.mode=3``-cc.inductor_compile_config='{"combo_kernels": false, "benchmark_combo_kernel": false}'`参数 - 加载W8A8量化模型启动服务时需要添加`-cc.mode=3``-cc.inductor_compile_config='{"combo_kernels": false, "benchmark_combo_kernel": false}'`参数
#### 单机推理 #### 单机推理
```bash ```bash
...@@ -88,7 +88,8 @@ curl http://localhost:8001/v1/chat/completions \ ...@@ -88,7 +88,8 @@ curl http://localhost:8001/v1/chat/completions \
"messages": [ "messages": [
{"role": "user", "content": "Type \"I love Qwen3.5\" backwards"} {"role": "user", "content": "Type \"I love Qwen3.5\" backwards"}
], ],
"temperature": 0.6 "temperature": 0.6,
"max_tokens": 512
}' }'
``` ```
...@@ -175,13 +176,14 @@ curl http://localhost:8001/v1/chat/completions \ ...@@ -175,13 +176,14 @@ curl http://localhost:8001/v1/chat/completions \
"messages": [ "messages": [
{"role": "user", "content": "Type \"I love Qwen3.5\" backwards"} {"role": "user", "content": "Type \"I love Qwen3.5\" backwards"}
], ],
"temperature": 0.6 "temperature": 0.6,
"max_tokens": 512
}' }'
``` ```
## 效果展示 ## 效果展示
<div align=center> <div align=center>
<img src="./doc/result-dcu.jpg"/> <img src="./doc/result-dcu.png"/>
</div> </div>
### 精度 ### 精度
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment