Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
NVIDIA-Nemotron-3_vllm
Commits
7c24d52b
Commit
7c24d52b
authored
Mar 20, 2026
by
raojy
💬
Browse files
Update README.md
parent
ebf6792b
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
8 deletions
+5
-8
README.md
README.md
+5
-8
No files found.
README.md
View file @
7c24d52b
...
@@ -22,14 +22,12 @@ Nemotron-3-Super-120B-A12B-BF16 是由英伟达 (NVIDIA) 训练的大语言模
...
@@ -22,14 +22,12 @@ Nemotron-3-Super-120B-A12B-BF16 是由英伟达 (NVIDIA) 训练的大语言模
| vllm | 0.15.1+das.opt1.alpha.dtk2604 |
| vllm | 0.15.1+das.opt1.alpha.dtk2604 |
| triton | 3.3.0+das.opt2.dtk2604.torch291.20260210.g1329924c |
| triton | 3.3.0+das.opt2.dtk2604.torch291.20260210.g1329924c |
| torch | 22.9.0+das.opt1.dtk2604.20260206.g275d08c2 |
| torch | 22.9.0+das.opt1.dtk2604.20260206.g275d08c2 |
|
torch
| 1.26.1 |
|
numpy
| 1.26.1 |
推荐使用
镜像:
`harbor.sourcefind.cn:5443/dcu/admin/base/custom:vllm0.15.1-ubuntu22.04-dtk26.04-0130-py3.10-20260220`
当前仅支持定制
镜像:
`harbor.sourcefind.cn:5443/dcu/admin/base/custom:vllm0.15.1-ubuntu22.04-dtk26.04-0130-py3.10-20260220`
-
挂载地址
`-v`
根据实际模型情况修改
-
挂载地址
`-v`
根据实际模型情况修改
Bash
```
```
docker run -it --shm-size 200g \
docker run -it --shm-size 200g \
--network=host \
--network=host \
...
@@ -45,6 +43,8 @@ docker run -it --shm-size 200g \
...
@@ -45,6 +43,8 @@ docker run -it --shm-size 200g \
-v /opt/hyhal/:/opt/hyhal/:ro \
-v /opt/hyhal/:/opt/hyhal/:ro \
harbor.sourcefind.cn:5443/dcu/admin/base/custom:vllm0.15.1-ubuntu22.04-dtk26.04-0130-py3.10-20260220 bash
harbor.sourcefind.cn:5443/dcu/admin/base/custom:vllm0.15.1-ubuntu22.04-dtk26.04-0130-py3.10-20260220 bash
```
```
更多镜像可前往
[
光源
](
https://sourcefind.cn/#/service-list
)
下载使用。
关于本项目 DCU 显卡所需的特殊深度学习库,numpy、vllm 库需要替换安装:
关于本项目 DCU 显卡所需的特殊深度学习库,numpy、vllm 库需要替换安装:
```
```
...
@@ -66,10 +66,7 @@ pip install numpy==1.26.1
...
@@ -66,10 +66,7 @@ pip install numpy==1.26.1
### vllm
### vllm
#### 单机推理(建议 8 卡)
#### 单机推理
**注意**
:对于 120B 参数量的 BF16 模型,单机推理建议至少使用 8 张 K100 AI。使用时需添加
`--disable-custom-all-reduce`
参数。
```
```
## serve启动
## serve启动
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment