Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
LLaMA_Fastchat_pytorch
Commits
9c5beeac
Commit
9c5beeac
authored
Oct 10, 2023
by
“yuguo”
Browse files
update
parent
d6b60084
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
9 additions
and
4 deletions
+9
-4
README.md
README.md
+9
-4
No files found.
README.md
View file @
9c5beeac
...
@@ -47,9 +47,7 @@ LLaMA,这是一个基础语言模型的集合,参数范围从7B到65B。在
...
@@ -47,9 +47,7 @@ LLaMA,这是一个基础语言模型的集合,参数范围从7B到65B。在
$ tree ./FastChat-main/playground/data
$ tree ./FastChat-main/playground/data
── alpaca-data-conversation.json
── alpaca-data-conversation.json
## LLAMA-13B微调(使用mpi)
## 环境配置
### 环境配置
按照节点环境修改env.sh,环境变量参考dtk-22.10。修改2节点16卡Z00L裸金属节点,要求dtk环境正常,mpirun文件夹下包含预编译好的openmpi库mpi4.tar.gz,可直接使用。关于本项目DCU显卡所需torch库等均可从
[
光合
](
https://developer.hpccube.com/tool/
)
开发者社区下载安装:
按照节点环境修改env.sh,环境变量参考dtk-22.10。修改2节点16卡Z00L裸金属节点,要求dtk环境正常,mpirun文件夹下包含预编译好的openmpi库mpi4.tar.gz,可直接使用。关于本项目DCU显卡所需torch库等均可从
[
光合
](
https://developer.hpccube.com/tool/
)
开发者社区下载安装:
...
@@ -64,9 +62,16 @@ cd ..
...
@@ -64,9 +62,16 @@ cd ..
pip3 install torch-1.10.0a0+git2040069.dtk2210-cp38-cp38-manylinux2014_x86_64.whl
pip3 install torch-1.10.0a0+git2040069.dtk2210-cp38-cp38-manylinux2014_x86_64.whl
pip3 install deepspeed-0.6.3+1b2721a.dtk2210-cp38-cp38-manylinux2014_x86_64.whl
pip3 install deepspeed-0.6.3+1b2721a.dtk2210-cp38-cp38-manylinux2014_x86_64.whl
pip3 install apex-0.1+gitdb7007a.dtk2210-cp38-cp38-manylinux2014_x86_64.whl(可选)
pip3 install apex-0.1+gitdb7007a.dtk2210-cp38-cp38-manylinux2014_x86_64.whl(可选)
pip3 uninstall wandb
```
```
### 训练
## 训练
权重链接
13B:
[
decapoda-research/llama-13b-hf · Hugging Face
](
https://huggingface.co/decapoda-research/llama-13b-hf
)
7B:
[
decapoda-research/llama-7b-hf · Hugging Face
](
https://huggingface.co/decapoda-research/llama-7b-hf
)
该训练脚本需要2节点,每节点8张DCU-Z100L-32G。按需更改mpi_single.sh中模型权重所在路径。
该训练脚本需要2节点,每节点8张DCU-Z100L-32G。按需更改mpi_single.sh中模型权重所在路径。
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment