Commit a8dfa799 authored by chenych's avatar chenych
Browse files

Change SCNET link and modify README

parent 60a524ee
......@@ -23,8 +23,6 @@ Llama在长文本能力上也取得了突破,具有超大的上下文窗口长
### Docker(方法一)
```bash
cd docker
docker build --no-cache -t llama-factory:latest .
docker run -it --shm-size 200g --network=host --name {docker_name} --privileged --device=/dev/kfd --device=/dev/dri --device=/dev/mkfd --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -u root -v /path/your_code_data/:/path/your_code_data/ -v /opt/hyhal/:/opt/hyhal/:ro {imageID} bash
cd /your_code_path/llama4_pytorch
......@@ -66,11 +64,12 @@ git clone https://developer.sourcefind.cn/codes/OpenDAS/llama-factory
SFT训练脚本示例,参考`llama-factory/train_full`下对应yaml文件。
**参数修改**
--model_name_or_path 修改为待训练模型地址,如 `/data/Llama-4-Scout-17B-16E-Instruct`
--dataset 微调训练集名称,可选数据集请参考 `llama-factory/data/dataset_info.json`
--template 将 default 修改为 `llama4`
--output_dir 模型保存地址
其他参数如:--learning_rate、--save_steps可根据自身硬件及需求进行修改。
- **--model_name_or_path**: 修改为待训练模型地址,如 `/data/Llama-4-Scout-17B-16E-Instruct`
- **--dataset**: 微调训练集名称,可选数据集请参考 `llama-factory/data/dataset_info.json`
- **--template**: 将 default 修改为 `llama4`
- **--output_dir**: 模型保存地址
其他参数如:`--learning_rate``--save_steps`可根据自身硬件及需求进行修改。
#### lora微调
......@@ -91,7 +90,6 @@ python infer_transformers.py --model_id /path_of/model_id
<img src="./doc/transformers_results.jpg"/>
</div>
### 精度
暂无
......@@ -103,11 +101,10 @@ python infer_transformers.py --model_id /path_of/model_id
制造,广媒,家居,教育
## 预训练权重
通过[SCNet AI社区模型库](https://www.scnet.cn/ui/aihub/models)下载预训练模型:
- [Llama-4-Scout-17B-16E](https://www.scnet.cn/ui/aihub/models/openaimodels/Llama-4-Scout-17B-16E)
- [Llama-4-Scout-17B-16E-Instruct](https://www.scnet.cn/ui/aihub/models/openaimodels/Llama-4-Scout-17B-16E-Instruct)
- [Llama-4-Maverick-17B-128E](https://www.scnet.cn/ui/aihub/models/openaimodels/Llama-4-Maverick-17B-128E)
- [Llama-4-Maverick-17B-128E-Instruct](https://www.scnet.cn/ui/aihub/models/openaimodels/Llama-4-Maverick-17B-128E-Instruct)
- [Llama-4-Scout-17B-16E](https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E)
- [Llama-4-Scout-17B-16E-Instruct](https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct)
- [Llama-4-Maverick-17B-128E](https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E)
- [Llama-4-Maverick-17B-128E-Instruct](https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Instruct)
## 源码仓库及问题反馈
- https://developer.hpccube.com/codes/modelzoo/llama4_pytorch
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment