Commit 126ffe1b authored by chenych's avatar chenych
Browse files

Update docker

parent 80e58670
Pipeline #3088 failed with stages
in 0 seconds
......@@ -54,6 +54,11 @@ pip uninstall vllm
pip install vllm-0.9.2+das.opt1.dtk25042.20251202.g9f9886d8-cp310-cp310-manylinux_2_28_x86_64.whl
```
**其他库安装方法**
```bash
pip install -r requirements.txt
```
## 数据集
......@@ -68,7 +73,7 @@ pip install vllm-0.9.2+das.opt1.dtk25042.20251202.g9f9886d8-cp310-cp310-manylinu
# fp8转bf16
python inference/fp8_cast_bf16.py --input-fp8-hf-path /path/to/DeepSeek-V3.2 --output-bf16-hf-path /path/to/DeepSeek-V3.2-bf16
```
转换完成后,将原模型中的 `generation_config.json`, `tokenizer_config.json`, `tokenizer.json`拷贝到`/path/to/DeepSeek-V3.2-Exp-bf16`中。
转换完成后,将原模型中的 `generation_config.json`, `tokenizer_config.json`, `tokenizer.json`拷贝到`/path/to/DeepSeek-V3.2-bf16`中。
拷贝config文件
```bash
......@@ -92,6 +97,7 @@ cp inference/config.json /path/to/DeepSeek-V3.2-bf16
</div>
```bash
export VLLM_USE_V32_ENCODE=1
export ALLREDUCE_STREAM_WITH_COMPUTE=1
export VLLM_HOST_IP=x.x.x.x # 对应计算节点的IP,建议选择IB口SOCKET_IFNAME对应IP地址
export HIP_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
......
cbor2
setproctitle
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment