Commit 8efe47a6 authored by zhaoying1's avatar zhaoying1
Browse files

update

parent 07bcc29c
......@@ -61,7 +61,7 @@ conda create -n chatglm python=3.8
3. 其它依赖库参照requirements.txt安装:
```
pip install -r requirements.txt --no-dependencies -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
pip install -r requirements.txt -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
```
## 数据集
......@@ -150,6 +150,7 @@ python src/cli_demo.py \
--finetuning_type lora \
--checkpoint_dir path_to_checkpoint
```
注意:对于所有“基座”(Base)模型,--template 参数可以是 default 或者 baichuan任意值。但“对话”(Chat)模型请务必使用baichuan。
### 导出微调后的模型
导出模型为huggingface格式:
......@@ -199,7 +200,10 @@ CUDA_VISIBLE_DEVICES=0 python src/train_bash.py \
```
## Result
<div align="center">
<img src="./assets/baichuan-result.png" width="500" height="100">
</div>
## 精度
- 以下为我们基于baichuan-13b-base模型进行全参数指令微调测试的loss收敛情况:
......
......@@ -2,7 +2,7 @@ transformers==4.31.0
datasets>=2.12.0
accelerate>=0.21.0
peft==0.4.0
trl>=0.7.1
trl==0.7.2
scipy
sentencepiece
tiktoken
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment