Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
llama3_pytorch
Commits
82e6800e
"docs/source/ko/optimization/torch2.0.md" did not exist on "e828232780554d54bdb527d3390fab0be042b72a"
Commit
82e6800e
authored
May 22, 2024
by
Rayyyyy
Browse files
Update xtuner in README
parent
06c5529f
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
4 deletions
+5
-4
README.md
README.md
+5
-4
No files found.
README.md
View file @
82e6800e
...
...
@@ -15,7 +15,6 @@ Llama-3中选择了一个相对标准的decoder-only的transformer架构。与Ll
**注意**
:bitsandbytes库功能不全,暂不支持量化相关
### Docker(方法一)
```
bash
docker pull image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.1.0-centos7.6-dtk24.04-py310
docker run
-it
-v
/path/your_code_data/:/path/your_code_data/
-v
/opt/hyhal/:/opt/hyhal/:ro
--shm-size
=
80G
--privileged
=
true
--device
=
/dev/kfd
--device
=
/dev/dri/
--group-add
video
--name
docker_name imageID bash
...
...
@@ -25,7 +24,6 @@ pip install -e .
```
### Dockerfile(方法二)
```
bash
cd
docker
docker build
--no-cache
-t
llama3:latest
.
...
...
@@ -62,9 +60,12 @@ pip install -e .
### xtuner微调方法
1.
训练库安装,请注意所需库版本
```
bash
pip uninstall fflash-attn
# 2.0.4+82379d7.abi0.dtk2404.torch2.1
pip uninstall flash-attn
# 2.0.4+82379d7.abi0.dtk2404.torch2.1
# docker环境含有deepspeed的可不进行安装, 需要对照版本是否一致即可
pip
install
deepspeed-0.12.3+das1.0+gita724046.abi0.dtk2404.torch2.1.0-cp310-cp310-manylinux2014_x86_64.whl
pip
install
-U
xtuner
# 0.1.18
git clone
-b
v0.1.18 https://github.com/InternLM/xtuner.git
cd
xtuner
pip
install
-e
'.[all]'
pip
install
mmengine
==
0.10.3
```
2.
下载预训练模型,具体模型请修改
`download_models.py`
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment