Commit e9c00134 authored by chenych's avatar chenych
Browse files

Change docker

parent 4a17b888
......@@ -23,7 +23,7 @@ Mistral Small 3.1 (25.03) 是一款多用途模型,专为编程、数学推理
### Docker(方法一)
```bash
docker pull image.sourcefind.cn:5000/dcu/admin/base/custom:vllm0.8.5-ubuntu22.04-dtk25.04-rc7-das1.5-py3.10-20250521-fixpy-rocblas0521-beta2
docker pull image.sourcefind.cn:5000/dcu/admin/base/custom:vllm0.8.5-ubuntu22.04-dtk25.04-rc7-das1.5-py3.10-20250612-fixpy-rocblas0611-rc2
docker run -it --shm-size 200g --network=host --name {docker_name} --privileged --device=/dev/kfd --device=/dev/dri --device=/dev/mkfd --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -u root -v /path/your_code_data/:/path/your_code_data/ -v /opt/hyhal/:/opt/hyhal/:ro {imageID} bash
cd /your_code_path/mistral-small_pytorch
......@@ -86,11 +86,6 @@ SFT训练脚本示例,参考`llama-factory/train_lora`下对应yaml文件。
## 推理
### vllm 推理
#### offline
需要先修改vllm代码:如图
<div align=center>
<img src="./doc/img_v3_02n6_c32e8a36-8338-4dff-9aec-d710746a61fg.jpg"/>
</div>
修改完成后,执行下面的方法。
```bash
python infer_vllm.py --model_name_or_path /path_of/model
......
FROM image.sourcefind.cn:5000/dcu/admin/base/custom:vllm0.8.5-ubuntu22.04-dtk25.04-rc7-das1.5-py3.10-20250521-fixpy-rocblas0521-beta2
\ No newline at end of file
FROM image.sourcefind.cn:5000/dcu/admin/base/custom:vllm0.8.5-ubuntu22.04-dtk25.04-rc7-das1.5-py3.10-20250612-fixpy-rocblas0611-rc2
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment