Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
GLM-5_vllm
Commits
fd1ada03
Commit
fd1ada03
authored
Apr 09, 2026
by
chenych
Browse files
Update README.
parent
8bb665c7
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
1 deletion
+4
-1
README.md
README.md
+4
-1
No files found.
README.md
View file @
fd1ada03
...
@@ -45,6 +45,8 @@ docker run -it \
...
@@ -45,6 +45,8 @@ docker run -it \
`暂无`
`暂无`
## 推理
## 推理
> 如果出现`ImportError: librocm_smi64.so.2: cannot open shaned object file: No such file or directory`报错,系机器hyhal版本较低所致,请进行升级。
### vllm
### vllm
#### 单机推理
#### 单机推理
1.
加入环境变量
1.
加入环境变量
...
@@ -171,7 +173,7 @@ ray start --address='x.x.x.x:6379' --num-gpus=8 --num-cpus=32
...
@@ -171,7 +173,7 @@ ray start --address='x.x.x.x:6379' --num-gpus=8 --num-cpus=32
vllm serve ZhipuAI/GLM-5
\
vllm serve ZhipuAI/GLM-5
\
--port
8001
\
--port
8001
\
--trust-remote-code
\
--trust-remote-code
\
--tensor-parallel-size
32
\
--tensor-parallel-size
32
\
# BW1000是32, BW1100是16
--gpu-memory-utilization
0.85
\
--gpu-memory-utilization
0.85
\
--distributed-executor-backend
ray
\
--distributed-executor-backend
ray
\
--dtype
bfloat16
\
--dtype
bfloat16
\
...
@@ -211,6 +213,7 @@ curl http://localhost:8001/v1/chat/completions \
...
@@ -211,6 +213,7 @@ curl http://localhost:8001/v1/chat/completions \
| 模型名称 | 权重大小 | DCU型号 | 最低卡数需求 |下载地址|
| 模型名称 | 权重大小 | DCU型号 | 最低卡数需求 |下载地址|
|:-----:|:----------:|:----------:|:---------------------:|:----------:|
|:-----:|:----------:|:----------:|:---------------------:|:----------:|
| GLM-5 | 744B | BW1000 | 32 |
[
ModelScope
](
https://modelscope.cn/models/ZhipuAI/GLM-5
)
|
| GLM-5 | 744B | BW1000 | 32 |
[
ModelScope
](
https://modelscope.cn/models/ZhipuAI/GLM-5
)
|
| GLM-5 | 744B | BW1100 | 16 |
[
ModelScope
](
https://modelscope.cn/models/ZhipuAI/GLM-5
)
|
| GLM-5-FP8 | 744B | BW1100 | 8 |
[
ModelScope
](
https://modelscope.cn/models/ZhipuAI/GLM-5-FP8
)
|
| GLM-5-FP8 | 744B | BW1100 | 8 |
[
ModelScope
](
https://modelscope.cn/models/ZhipuAI/GLM-5-FP8
)
|
## 源码仓库及问题反馈
## 源码仓库及问题反馈
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment