Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
Deepseek_Math_7b_pytorch
Commits
3cf08ace
Commit
3cf08ace
authored
Jul 01, 2024
by
zhougaofeng
Browse files
Update README.md
parent
dc14ba19
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
18 additions
and
11 deletions
+18
-11
README.md
README.md
+18
-11
No files found.
README.md
View file @
3cf08ace
...
@@ -19,13 +19,14 @@
...
@@ -19,13 +19,14 @@
```
```
docker pull image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.1.0-centos7.6-dtk24.04-py310
docker pull image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.1.0-centos7.6-dtk24.04-py310
docker run -it --shm-size=1024G -v
/parastor/home/deepseek-math-pytorch:/home/deepseek-math-pytorch
-v /opt/hyhal:/opt/hyhal --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name deepseek-math <your IMAGE ID> bash # <your IMAGE ID>为以上拉取的docker的镜像ID替换,本镜像为:c85ed27005f2
docker run -it --shm-size=1024G -v
<Host Path>:<Container Path>
-v /opt/hyhal:/opt/hyhal --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name deepseek-math <your IMAGE ID> bash # <your IMAGE ID>为以上拉取的docker的镜像ID替换,本镜像为:c85ed27005f2
cd /home/deepseek-math-pytorch
cd /home/deepseek-math-pytorch
pip install -r requirement.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
pip install -r requirement
s
.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
# deepspeed、bitsandbytes可从whl.zip文件里获取安装:
# deepspeed、bitsandbytes可从whl.zip文件里获取安装:
pip install deepspeed-0.12.3+das1.0+gita724046.abi0.dtk2404.torch2.1.0-cp310-cp310-manylinux2014_x86_64.whl
pip install bitsandbytes-0.42.0-py3-none-any.whl
pip install bitsandbytes-0.42.0-py3-none-any.whl
pip uninstall vllm
```
```
### Dockerfile(方法二)
### Dockerfile(方法二)
...
@@ -34,12 +35,12 @@ pip install bitsandbytes-0.42.0-py3-none-any.whl
...
@@ -34,12 +35,12 @@ pip install bitsandbytes-0.42.0-py3-none-any.whl
```
```
docker build -t deepseek-math-df:latest .
docker build -t deepseek-math-df:latest .
docker run -it --shm-size=1024G -v
/parastor/home/deepseek-math-pytorch:/home/deepseek-math-pytorch
-v /opt/hyhal:/opt/hyhal --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name deepseek-math deepseek-math-df bash
docker run -it --shm-size=1024G -v
<Host Path>:<Container Path>
-v /opt/hyhal:/opt/hyhal --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name deepseek-math deepseek-math-df bash
pip install -r requirement.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
pip install -r requirement
s
.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
# deepspeed、bitsandbytes可从whl.zip文件里获取安装:
# deepspeed、bitsandbytes可从whl.zip文件里获取安装:
pip install deepspeed-0.12.3+das1.0+gita724046.abi0.dtk2404.torch2.1.0-cp310-cp310-manylinux2014_x86_64.whl
pip install bitsandbytes-0.42.0-py3-none-any.whl
pip install bitsandbytes-0.42.0-py3-none-any.whl
pip uninstall vllm
```
```
### Anaconda(方法三)
### Anaconda(方法三)
...
@@ -49,7 +50,7 @@ pip install bitsandbytes-0.42.0-py3-none-any.whl
...
@@ -49,7 +50,7 @@ pip install bitsandbytes-0.42.0-py3-none-any.whl
关于本项目 DCU 显卡所需的特殊深度学习库可从
[
光合
](
https://developer.hpccube.com/tool/
)
开发者社区下载安装。
关于本项目 DCU 显卡所需的特殊深度学习库可从
[
光合
](
https://developer.hpccube.com/tool/
)
开发者社区下载安装。
```
```
DTK驱动:dtk2
3
.04
DTK驱动:dtk2
4
.04
python:python3.10
python:python3.10
torch: 2.1.0
torch: 2.1.0
deepspeed:0.12.3
deepspeed:0.12.3
...
@@ -63,6 +64,7 @@ triton:2.1.0
...
@@ -63,6 +64,7 @@ triton:2.1.0
```
```
pip install -r requirements.txt
pip install -r requirements.txt
pip uninstall vllm
```
```
## 数据集
## 数据集
...
@@ -77,7 +79,7 @@ pip install -r requirements.txt
...
@@ -77,7 +79,7 @@ pip install -r requirements.txt
## 训练
## 训练
一般情况下,ModelZoo 上的项目提供单机训练的启动方法即可,单机单卡、单机多卡至少提供其一训练方法。
根据实际路径修改模型、数据集路径
### 单机单卡
### 单机单卡
...
@@ -91,7 +93,6 @@ sh single_node.sh
...
@@ -91,7 +93,6 @@ sh single_node.sh
sh multi_node.sh
sh multi_node.sh
```
```
## 推理
## 推理
```
```
...
@@ -104,7 +105,7 @@ sh 7B_single_node.sh
...
@@ -104,7 +105,7 @@ sh 7B_single_node.sh
使用的加速卡:2张 DCU-K100-64G
使用的加速卡:2张 DCU-K100-64G
<div
align=
center
>
<div
align=
center
>
<img
src=
"./doc/
training_loss
.png"
/>
<img
src=
"./doc/
result
.png"
/>
</div>
</div>
### 精度
### 精度
...
@@ -126,10 +127,16 @@ sh 7B_single_node.sh
...
@@ -126,10 +127,16 @@ sh 7B_single_node.sh
`科研,教育,金融`
`科研,教育,金融`
### 预训练权重
预训练权重下载中心: huggingface[https://huggingface.co/deepseek-ai/deepseek-math-7b-base]
模型目录结构如下:
<div
align=
center
>
<img
src=
"./doc/model.png"
/>
</div>
## 源码仓库及问题反馈
## 源码仓库及问题反馈
-
https://
gith
ub.com/
deepseek-ai/D
eep
S
eek
-M
ath
-
https://
developer.hpcc
ub
e
.com/
codes/modelzoo/d
eep
s
eek
_m
ath
_7b_pytorch
## 参考资料
## 参考资料
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment