Commit 8efe47a6 authored by zhaoying1's avatar zhaoying1
Browse files

update

parent 07bcc29c
...@@ -61,7 +61,7 @@ conda create -n chatglm python=3.8 ...@@ -61,7 +61,7 @@ conda create -n chatglm python=3.8
3. 其它依赖库参照requirements.txt安装: 3. 其它依赖库参照requirements.txt安装:
``` ```
pip install -r requirements.txt --no-dependencies -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com pip install -r requirements.txt -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
``` ```
## 数据集 ## 数据集
...@@ -150,6 +150,7 @@ python src/cli_demo.py \ ...@@ -150,6 +150,7 @@ python src/cli_demo.py \
--finetuning_type lora \ --finetuning_type lora \
--checkpoint_dir path_to_checkpoint --checkpoint_dir path_to_checkpoint
``` ```
注意:对于所有“基座”(Base)模型,--template 参数可以是 default 或者 baichuan任意值。但“对话”(Chat)模型请务必使用baichuan。
### 导出微调后的模型 ### 导出微调后的模型
导出模型为huggingface格式: 导出模型为huggingface格式:
...@@ -199,7 +200,10 @@ CUDA_VISIBLE_DEVICES=0 python src/train_bash.py \ ...@@ -199,7 +200,10 @@ CUDA_VISIBLE_DEVICES=0 python src/train_bash.py \
``` ```
## Result ## Result
<div align="center">
<img src="./assets/baichuan-result.png" width="500" height="100">
</div>
## 精度 ## 精度
- 以下为我们基于baichuan-13b-base模型进行全参数指令微调测试的loss收敛情况: - 以下为我们基于baichuan-13b-base模型进行全参数指令微调测试的loss收敛情况:
......
...@@ -2,7 +2,7 @@ transformers==4.31.0 ...@@ -2,7 +2,7 @@ transformers==4.31.0
datasets>=2.12.0 datasets>=2.12.0
accelerate>=0.21.0 accelerate>=0.21.0
peft==0.4.0 peft==0.4.0
trl>=0.7.1 trl==0.7.2
scipy scipy
sentencepiece sentencepiece
tiktoken tiktoken
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment