Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
GLM-5.1
Commits
c5ba7b50
Commit
c5ba7b50
authored
Apr 09, 2026
by
chenych
Browse files
Update README.
parent
41e7937b
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
3 deletions
+5
-3
README.md
README.md
+5
-3
No files found.
README.md
View file @
c5ba7b50
...
...
@@ -51,6 +51,8 @@ docker run -it \
`暂无`
## 推理
> 如果出现`ImportError: librocm_smi64.so.2: cannot open shaned object file: No such file or directory`报错,系机器hyhal版本较低所致,请进行升级。
### vllm
#### 单机推理
1.
加入环境变量
...
...
@@ -58,7 +60,6 @@ docker run -it \
# 环境变量
rm
-rf
~/.cache
rm
-rf
~/.triton
export
VLLM_TORCH_PROFILER_DIR
=
/home/GLM-5/
export
HIP_VISIBLE_DEVICES
=
0,1,2,3,4,5,6,7
export
ALLREDUCE_STREAM_WITH_COMPUTE
=
1
export
NCCL_MIN_NCHANNELS
=
16
...
...
@@ -174,7 +175,7 @@ ray start --address='x.x.x.x:6379' --num-gpus=8 --num-cpus=32
vllm serve ZhipuAI/GLM-5.1
\
--port
8001
\
--trust-remote-code
\
--tensor-parallel-size
32
\
--tensor-parallel-size
32
\
# BW1000是32, BW1100是16
--gpu-memory-utilization
0.85
\
--distributed-executor-backend
ray
\
--dtype
bfloat16
\
...
...
@@ -211,7 +212,8 @@ curl http://localhost:8001/v1/chat/completions \
## 预训练权重
| 模型名称 | 权重大小 | DCU型号 | 最低卡数需求 |下载地址|
|:-----:|:----------:|:----------:|:---------------------:|:----------:|
| GLM-5.1 | 754B | BW1000,BW1100 | 32 |
[
ModelScope
](
https://modelscope.cn/models/ZhipuAI/GLM-5.1
)
|
| GLM-5.1 | 754B | BW1000 | 32 |
[
ModelScope
](
https://modelscope.cn/models/ZhipuAI/GLM-5.1
)
|
| GLM-5.1 | 754B | BW1100 | 16 |
[
ModelScope
](
https://modelscope.cn/models/ZhipuAI/GLM-5.1
)
|
| GLM-5.1-FP8 | 754B | BW1100 | 8 |
[
ModelScope
](
https://modelscope.cn/models/ZhipuAI/GLM-5.1-FP8
)
|
## 源码仓库及问题反馈
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment