Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
ModelZoo
Qwen3.5
Commits
7a751c0d
Commit
7a751c0d
authored
Feb 27, 2026
by
luopl
Browse files
add K100 AI
parent
d6e82889
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
25 additions
and
1 deletion
+25
-1
README.md
README.md
+25
-1
No files found.
README.md
View file @
7a751c0d
# Qwen3.5
_vllm
# Qwen3.5
## 论文
## 论文
[
Qwen3.5
](
https://qwen.ai/blog?id=qwen3.5
)
[
Qwen3.5
](
https://qwen.ai/blog?id=qwen3.5
)
...
@@ -58,6 +58,30 @@ pip install numpy==1.25.0
...
@@ -58,6 +58,30 @@ pip install numpy==1.25.0
## 推理
## 推理
### vllm
### vllm
#### 单机推理
**注意**
:使用
`K100 AI`
集群启动服务时需要添加
`--disable-custom-all-reduce`
参数
```
bash
## serve启动
vllm serve Qwen/Qwen3.5-35B-A3B
\
--port
8001
\
--tensor-parallel-size
16
\
--max-model-len
262144
\
--reasoning-parser
qwen3
## client访问
curl http://localhost:8001/v1/chat/completions
\
-H
"Content-Type: application/json"
\
-d
'{
"model": "Qwen/Qwen3.5-35B-A3B",
"messages": [
{"role": "user", "content": "Type \"I love Qwen3.5\" backwards"}
],
"temperature": 0.6
}'
```
#### 多机推理
#### 多机推理
1.
加入环境变量
1.
加入环境变量
> 请注意:
> 请注意:
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment