Commit 3cf08ace authored by zhougaofeng's avatar zhougaofeng
Browse files

Update README.md

parent dc14ba19
...@@ -19,13 +19,14 @@ ...@@ -19,13 +19,14 @@
``` ```
docker pull image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.1.0-centos7.6-dtk24.04-py310 docker pull image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.1.0-centos7.6-dtk24.04-py310
docker run -it --shm-size=1024G -v /parastor/home/deepseek-math-pytorch:/home/deepseek-math-pytorch -v /opt/hyhal:/opt/hyhal --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name deepseek-math <your IMAGE ID> bash # <your IMAGE ID>为以上拉取的docker的镜像ID替换,本镜像为:c85ed27005f2 docker run -it --shm-size=1024G -v <Host Path>:<Container Path> -v /opt/hyhal:/opt/hyhal --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name deepseek-math <your IMAGE ID> bash # <your IMAGE ID>为以上拉取的docker的镜像ID替换,本镜像为:c85ed27005f2
cd /home/deepseek-math-pytorch cd /home/deepseek-math-pytorch
pip install -r requirement.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
# deepspeed、bitsandbytes可从whl.zip文件里获取安装: # deepspeed、bitsandbytes可从whl.zip文件里获取安装:
pip install deepspeed-0.12.3+das1.0+gita724046.abi0.dtk2404.torch2.1.0-cp310-cp310-manylinux2014_x86_64.whl
pip install bitsandbytes-0.42.0-py3-none-any.whl pip install bitsandbytes-0.42.0-py3-none-any.whl
pip uninstall vllm
``` ```
### Dockerfile(方法二) ### Dockerfile(方法二)
...@@ -34,12 +35,12 @@ pip install bitsandbytes-0.42.0-py3-none-any.whl ...@@ -34,12 +35,12 @@ pip install bitsandbytes-0.42.0-py3-none-any.whl
``` ```
docker build -t deepseek-math-df:latest . docker build -t deepseek-math-df:latest .
docker run -it --shm-size=1024G -v /parastor/home/deepseek-math-pytorch:/home/deepseek-math-pytorch -v /opt/hyhal:/opt/hyhal --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name deepseek-math deepseek-math-df bash docker run -it --shm-size=1024G -v <Host Path>:<Container Path> -v /opt/hyhal:/opt/hyhal --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name deepseek-math deepseek-math-df bash
pip install -r requirement.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
# deepspeed、bitsandbytes可从whl.zip文件里获取安装: # deepspeed、bitsandbytes可从whl.zip文件里获取安装:
pip install deepspeed-0.12.3+das1.0+gita724046.abi0.dtk2404.torch2.1.0-cp310-cp310-manylinux2014_x86_64.whl
pip install bitsandbytes-0.42.0-py3-none-any.whl pip install bitsandbytes-0.42.0-py3-none-any.whl
pip uninstall vllm
``` ```
### Anaconda(方法三) ### Anaconda(方法三)
...@@ -49,7 +50,7 @@ pip install bitsandbytes-0.42.0-py3-none-any.whl ...@@ -49,7 +50,7 @@ pip install bitsandbytes-0.42.0-py3-none-any.whl
关于本项目 DCU 显卡所需的特殊深度学习库可从[光合](https://developer.hpccube.com/tool/)开发者社区下载安装。 关于本项目 DCU 显卡所需的特殊深度学习库可从[光合](https://developer.hpccube.com/tool/)开发者社区下载安装。
``` ```
DTK驱动:dtk23.04 DTK驱动:dtk24.04
python:python3.10 python:python3.10
torch: 2.1.0 torch: 2.1.0
deepspeed:0.12.3 deepspeed:0.12.3
...@@ -63,6 +64,7 @@ triton:2.1.0 ...@@ -63,6 +64,7 @@ triton:2.1.0
``` ```
pip install -r requirements.txt pip install -r requirements.txt
pip uninstall vllm
``` ```
## 数据集 ## 数据集
...@@ -77,7 +79,7 @@ pip install -r requirements.txt ...@@ -77,7 +79,7 @@ pip install -r requirements.txt
## 训练 ## 训练
一般情况下,ModelZoo 上的项目提供单机训练的启动方法即可,单机单卡、单机多卡至少提供其一训练方法。 根据实际路径修改模型、数据集路径
### 单机单卡 ### 单机单卡
...@@ -91,7 +93,6 @@ sh single_node.sh ...@@ -91,7 +93,6 @@ sh single_node.sh
sh multi_node.sh sh multi_node.sh
``` ```
## 推理 ## 推理
``` ```
...@@ -104,7 +105,7 @@ sh 7B_single_node.sh ...@@ -104,7 +105,7 @@ sh 7B_single_node.sh
使用的加速卡:2张 DCU-K100-64G 使用的加速卡:2张 DCU-K100-64G
<div align=center> <div align=center>
<img src="./doc/training_loss.png"/> <img src="./doc/result.png"/>
</div> </div>
### 精度 ### 精度
...@@ -126,10 +127,16 @@ sh 7B_single_node.sh ...@@ -126,10 +127,16 @@ sh 7B_single_node.sh
`科研,教育,金融` `科研,教育,金融`
### 预训练权重
预训练权重下载中心: huggingface[https://huggingface.co/deepseek-ai/deepseek-math-7b-base]
模型目录结构如下:
<div align=center>
<img src="./doc/model.png"/>
</div>
## 源码仓库及问题反馈 ## 源码仓库及问题反馈
- https://github.com/deepseek-ai/DeepSeek-Math - https://developer.hpccube.com/codes/modelzoo/deepseek_math_7b_pytorch
## 参考资料 ## 参考资料
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment