Commit c5dc70c5 authored by Rayyyyy's avatar Rayyyyy
Browse files

Modify infos in README

parent 86913514
......@@ -71,9 +71,8 @@ pip install mmengine==0.10.3
cd /your_code_path/llama3_pytorch
pip install modelscope
python download_models.py
mv ./LLM-Research/* ./
```
2. 修改[llama3_8b_instruct_qlora_alpaca_e3_M.py](./llama3_8b_instruct_qlora_alpaca_e3_M.py)代码中的`pretrained_model_name_or_path``data_path`为本地对应数据地址;
2. 修改[llama3_8b_instruct_qlora_alpaca_e3_M.py](./llama3_8b_instruct_qlora_alpaca_e3_M.py)代码中的`pretrained_model_name_or_path``data_path`为本地模型、数据地址;
3. 根据硬件环境和自身训练需求来调整`max_length``batch_size``accumulative_counts``max_epochs``lr``save_steps``evaluation_freq`、model.lora中的`r``lora_alpha`参数,默认参数支持4*32G;
4. ${DCU_NUM}参数修改为要使用的DCU卡数量,不同数据集需要修改llama3_8b_instruct_qlora_alpaca_e3_M.py中`SYSTEM``evaluation_inputs``dataset_map_fn``train_dataloader.sampler``train_cfg`参数设置,详情请参考代码注释项,当前默认alpaca数据集。
5. 执行
......
......@@ -24,7 +24,7 @@ from xtuner.utils import PROMPT_TEMPLATE, SYSTEM_TEMPLATE
# PART 1 Settings #
#######################################################################
# Model
pretrained_model_name_or_path = '/home/llama3/Meta-Llama-3-8B-Instruct'
pretrained_model_name_or_path = '/your_model_save_path/LLM-Research/Meta-Llama-3-8B-Instruct'
use_varlen_attn = False # new
# Data
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment