Commit 3cf08ace authored by zhougaofeng's avatar zhougaofeng
Browse files

Update README.md

parent dc14ba19
......@@ -19,13 +19,14 @@
```
docker pull image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.1.0-centos7.6-dtk24.04-py310
docker run -it --shm-size=1024G -v /parastor/home/deepseek-math-pytorch:/home/deepseek-math-pytorch -v /opt/hyhal:/opt/hyhal --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name deepseek-math <your IMAGE ID> bash # <your IMAGE ID>为以上拉取的docker的镜像ID替换,本镜像为:c85ed27005f2
docker run -it --shm-size=1024G -v <Host Path>:<Container Path> -v /opt/hyhal:/opt/hyhal --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name deepseek-math <your IMAGE ID> bash # <your IMAGE ID>为以上拉取的docker的镜像ID替换,本镜像为:c85ed27005f2
cd /home/deepseek-math-pytorch
pip install -r requirement.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
# deepspeed、bitsandbytes可从whl.zip文件里获取安装:
pip install deepspeed-0.12.3+das1.0+gita724046.abi0.dtk2404.torch2.1.0-cp310-cp310-manylinux2014_x86_64.whl
pip install bitsandbytes-0.42.0-py3-none-any.whl
pip uninstall vllm
```
### Dockerfile(方法二)
......@@ -34,12 +35,12 @@ pip install bitsandbytes-0.42.0-py3-none-any.whl
```
docker build -t deepseek-math-df:latest .
docker run -it --shm-size=1024G -v /parastor/home/deepseek-math-pytorch:/home/deepseek-math-pytorch -v /opt/hyhal:/opt/hyhal --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name deepseek-math deepseek-math-df bash
pip install -r requirement.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
docker run -it --shm-size=1024G -v <Host Path>:<Container Path> -v /opt/hyhal:/opt/hyhal --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name deepseek-math deepseek-math-df bash
pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
# deepspeed、bitsandbytes可从whl.zip文件里获取安装:
pip install deepspeed-0.12.3+das1.0+gita724046.abi0.dtk2404.torch2.1.0-cp310-cp310-manylinux2014_x86_64.whl
pip install bitsandbytes-0.42.0-py3-none-any.whl
pip uninstall vllm
```
### Anaconda(方法三)
......@@ -49,7 +50,7 @@ pip install bitsandbytes-0.42.0-py3-none-any.whl
关于本项目 DCU 显卡所需的特殊深度学习库可从[光合](https://developer.hpccube.com/tool/)开发者社区下载安装。
```
DTK驱动:dtk23.04
DTK驱动:dtk24.04
python:python3.10
torch: 2.1.0
deepspeed:0.12.3
......@@ -63,6 +64,7 @@ triton:2.1.0
```
pip install -r requirements.txt
pip uninstall vllm
```
## 数据集
......@@ -77,7 +79,7 @@ pip install -r requirements.txt
## 训练
一般情况下,ModelZoo 上的项目提供单机训练的启动方法即可,单机单卡、单机多卡至少提供其一训练方法。
根据实际路径修改模型、数据集路径
### 单机单卡
......@@ -91,7 +93,6 @@ sh single_node.sh
sh multi_node.sh
```
## 推理
```
......@@ -104,7 +105,7 @@ sh 7B_single_node.sh
使用的加速卡:2张 DCU-K100-64G
<div align=center>
<img src="./doc/training_loss.png"/>
<img src="./doc/result.png"/>
</div>
### 精度
......@@ -126,10 +127,16 @@ sh 7B_single_node.sh
`科研,教育,金融`
### 预训练权重
预训练权重下载中心: huggingface[https://huggingface.co/deepseek-ai/deepseek-math-7b-base]
模型目录结构如下:
<div align=center>
<img src="./doc/model.png"/>
</div>
## 源码仓库及问题反馈
- https://github.com/deepseek-ai/DeepSeek-Math
- https://developer.hpccube.com/codes/modelzoo/deepseek_math_7b_pytorch
## 参考资料
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment