Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
DeepSeek-V3.2_vllm
Commits
126ffe1b
Commit
126ffe1b
authored
Dec 04, 2025
by
chenych
Browse files
Update docker
parent
80e58670
Pipeline
#3088
failed with stages
in 0 seconds
Changes
3
Pipelines
2
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
9 additions
and
1 deletion
+9
-1
README.md
README.md
+7
-1
inference/__pycache__/kernel.cpython-310.pyc
inference/__pycache__/kernel.cpython-310.pyc
+0
-0
requirements.txt
requirements.txt
+2
-0
No files found.
README.md
View file @
126ffe1b
...
...
@@ -54,6 +54,11 @@ pip uninstall vllm
pip
install
vllm-0.9.2+das.opt1.dtk25042.20251202.g9f9886d8-cp310-cp310-manylinux_2_28_x86_64.whl
```
**其他库安装方法**
```
bash
pip
install
-r
requirements.txt
```
## 数据集
无
...
...
@@ -68,7 +73,7 @@ pip install vllm-0.9.2+das.opt1.dtk25042.20251202.g9f9886d8-cp310-cp310-manylinu
# fp8转bf16
python inference/fp8_cast_bf16.py
--input-fp8-hf-path
/path/to/DeepSeek-V3.2
--output-bf16-hf-path
/path/to/DeepSeek-V3.2-bf16
```
转换完成后,将原模型中的
`generation_config.json`
,
`tokenizer_config.json`
,
`tokenizer.json`
拷贝到
`/path/to/DeepSeek-V3.2-
Exp-
bf16`
中。
转换完成后,将原模型中的
`generation_config.json`
,
`tokenizer_config.json`
,
`tokenizer.json`
拷贝到
`/path/to/DeepSeek-V3.2-bf16`
中。
拷贝config文件
```
bash
...
...
@@ -92,6 +97,7 @@ cp inference/config.json /path/to/DeepSeek-V3.2-bf16
</div>
```
bash
export
VLLM_USE_V32_ENCODE
=
1
export
ALLREDUCE_STREAM_WITH_COMPUTE
=
1
export
VLLM_HOST_IP
=
x.x.x.x
# 对应计算节点的IP,建议选择IB口SOCKET_IFNAME对应IP地址
export
HIP_VISIBLE_DEVICES
=
0,1,2,3,4,5,6,7
...
...
inference/__pycache__/kernel.cpython-310.pyc
deleted
100644 → 0
View file @
80e58670
File deleted
requirements.txt
0 → 100644
View file @
126ffe1b
cbor2
setproctitle
\ No newline at end of file
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment