Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
Llama4_pytorch
Commits
a8dfa799
Commit
a8dfa799
authored
Apr 14, 2025
by
chenych
Browse files
Change SCNET link and modify README
parent
60a524ee
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
10 additions
and
13 deletions
+10
-13
README.md
README.md
+10
-13
No files found.
README.md
View file @
a8dfa799
...
...
@@ -23,8 +23,6 @@ Llama在长文本能力上也取得了突破,具有超大的上下文窗口长
### Docker(方法一)
```
bash
cd
docker
docker build
--no-cache
-t
llama-factory:latest
.
docker run
-it
--shm-size
200g
--network
=
host
--name
{
docker_name
}
--privileged
--device
=
/dev/kfd
--device
=
/dev/dri
--device
=
/dev/mkfd
--group-add
video
--cap-add
=
SYS_PTRACE
--security-opt
seccomp
=
unconfined
-u
root
-v
/path/your_code_data/:/path/your_code_data/
-v
/opt/hyhal/:/opt/hyhal/:ro
{
imageID
}
bash
cd
/your_code_path/llama4_pytorch
...
...
@@ -66,11 +64,12 @@ git clone https://developer.sourcefind.cn/codes/OpenDAS/llama-factory
SFT训练脚本示例,参考
`llama-factory/train_full`
下对应yaml文件。
**参数修改**
:
--model_name_or_path 修改为待训练模型地址,如
`/data/Llama-4-Scout-17B-16E-Instruct`
--dataset 微调训练集名称,可选数据集请参考
`llama-factory/data/dataset_info.json`
--template 将 default 修改为
`llama4`
--output_dir 模型保存地址
其他参数如:--learning_rate、--save_steps可根据自身硬件及需求进行修改。
-
**--model_name_or_path**
: 修改为待训练模型地址,如
`/data/Llama-4-Scout-17B-16E-Instruct`
-
**--dataset**
: 微调训练集名称,可选数据集请参考
`llama-factory/data/dataset_info.json`
-
**--template**
: 将 default 修改为
`llama4`
-
**--output_dir**
: 模型保存地址
其他参数如:
`--learning_rate`
、
`--save_steps`
可根据自身硬件及需求进行修改。
#### lora微调
...
...
@@ -91,7 +90,6 @@ python infer_transformers.py --model_id /path_of/model_id
<img
src=
"./doc/transformers_results.jpg"
/>
</div>
### 精度
暂无
...
...
@@ -103,11 +101,10 @@ python infer_transformers.py --model_id /path_of/model_id
制造,广媒,家居,教育
## 预训练权重
通过
[
SCNet AI社区模型库
](
https://www.scnet.cn/ui/aihub/models
)
下载预训练模型:
-
[
Llama-4-Scout-17B-16E
](
https://www.scnet.cn/ui/aihub/models/openaimodels/Llama-4-Scout-17B-16E
)
-
[
Llama-4-Scout-17B-16E-Instruct
](
https://www.scnet.cn/ui/aihub/models/openaimodels/Llama-4-Scout-17B-16E-Instruct
)
-
[
Llama-4-Maverick-17B-128E
](
https://www.scnet.cn/ui/aihub/models/openaimodels/Llama-4-Maverick-17B-128E
)
-
[
Llama-4-Maverick-17B-128E-Instruct
](
https://www.scnet.cn/ui/aihub/models/openaimodels/Llama-4-Maverick-17B-128E-Instruct
)
-
[
Llama-4-Scout-17B-16E
](
https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E
)
-
[
Llama-4-Scout-17B-16E-Instruct
](
https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct
)
-
[
Llama-4-Maverick-17B-128E
](
https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E
)
-
[
Llama-4-Maverick-17B-128E-Instruct
](
https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Instruct
)
## 源码仓库及问题反馈
-
https://developer.hpccube.com/codes/modelzoo/llama4_pytorch
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment