README.md 8.92 KB
Newer Older
zhaoying1's avatar
zhaoying1 committed
1
2
# ChatGLM-6B

zhaoying1's avatar
zhaoying1 committed
3
4
5
## 论文
`GLM: General Language Model Pretraining with Autoregressive Blank Infilling`
- [https://arxiv.org/abs/2103.10360](https://arxiv.org/abs/2103.10360)
zhaoying1's avatar
zhaoying1 committed
6

zhaoying1's avatar
zhaoying1 committed
7
## 模型结构
zhaoying1's avatar
zhaoying1 committed
8
ChatGLM-6B 是清华大学开源的开源的、支持中英双语的对话语言模型,基于 [General Language Model (GLM)](https://github.com/THUDM/GLM) 架构,具有 62 亿参数。ChatGLM-6B 使用了和 ChatGPT 相似的技术,针对中文问答和对话进行了优化。经过约 1T 标识符的中英双语训练,辅以监督微调、反馈自助、人类反馈强化学习等技术的加持,62 亿参数的 ChatGLM-6B 已经能生成相当符合人类偏好的回答。
zhaoying1's avatar
zhaoying1 committed
9

zhaoying1's avatar
zhaoying1 committed
10
11
12
13
14
15
16
17
18
19

以下是ChatGLM-6B的主要网络参数配置:


| 模型名称 | 隐含层维度 | 层数 | 头数 | 词表大小 | 位置编码 | 最大长 |
| -------- | -------- | -------- | -------- | -------- | -------- | -------- | 
|ChatGLM-6B | 4,096 | 28 | 32 | 130528 |  RoPE | 2048 |

## 算法原理
ChatGLM-6B基于GLM架构开发。GLM是一种基于Transformer的语言模型,以自回归空白填充为训练目标, 同时具备自回归和自编码能力。
zhaoying1's avatar
zhaoying1 committed
20

zhaoying1's avatar
zhaoying1 committed
21
22
23
<div align="center">
<img src="ptuning/media/GLM.png" width="550" height="200">
</div>
zhaoying1's avatar
zhaoying1 committed
24
25

## 环境配置
yuguo960516yuguo's avatar
readme  
yuguo960516yuguo committed
26

zhaoying1's avatar
zhaoying1 committed
27
### Docker(方式一)
zhaoying1's avatar
zhaoying1 committed
28
29
30
31
32
33
34
推荐使用docker方式运行,提供拉取的docker镜像:
```
docker pull image.sourcefind.cn:5000/dcu/admin/base/pytorch:1.13.1-centos7.6-dtk-23.04-py38-latest
```

进入docker,安装docker中没有的依赖:
```
zhaoying1's avatar
zhaoying1 committed
35
docker run -dit --network=host --name=chatglm --privileged --device=/dev/kfd --device=/dev/dri --ipc=host --shm-size=16G  --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -u root --ulimit stack=-1:-1 --ulimit memlock=-1:-1 image.sourcefind.cn:5000/dcu/admin/base/pytorch:1.13.1-centos7.6-dtk-23.04-py38-latest
zhaoying1's avatar
zhaoying1 committed
36
pip install transformers==4.28.0 -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
zhaoying1's avatar
zhaoying1 committed
37
pip install accelerate sentencepiece mdtex2html gradio rouge_chinese nltk jieba datasets protobuf peft -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
zhaoying1's avatar
zhaoying1 committed
38
```
zhaoying1's avatar
zhaoying1 committed
39
40
41
42
43
44
### Dockerfile(方式二)
```
docker build -t chatglm:latest .
docker run -dit --network=host --name=baichuan --privileged --device=/dev/kfd --device=/dev/dri --ipc=host --shm-size=16G  --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -u root --ulimit stack=-1:-1 --ulimit memlock=-1:-1 baichuan:latest
docker exec -it baichuan /bin/bash
```
zhaoying1's avatar
zhaoying1 committed
45

zhaoying1's avatar
zhaoying1 committed
46
47
48
49
50
### Conda(方法三)
1. 创建conda虚拟环境:
```
conda create -n chatglm python=3.8
```
zhaoying1's avatar
zhaoying1 committed
51

zhaoying1's avatar
zhaoying1 committed
52
53
54
55
2. 关于本项目DCU显卡所需的工具包、深度学习库等均可从[光合](https://developer.hpccube.com/tool/)开发者社区下载安装。
- [DTK 23.04](https://cancon.hpccube.com:65024/1/main/DTK-23.04.1)
- [Pytorch 1.13.1](https://cancon.hpccube.com:65024/4/main/pytorch/dtk23.04)
- [Deepspeed 0.9.2](https://cancon.hpccube.com:65024/4/main/deepspeed/dtk23.04)
zhaoying1's avatar
zhaoying1 committed
56

zhaoying1's avatar
zhaoying1 committed
57
    Tips:以上dtk驱动、python、deepspeed等工具版本需要严格一一对应。
zhaoying1's avatar
zhaoying1 committed
58

zhaoying1's avatar
zhaoying1 committed
59
60
61
62
3. 其它依赖库参照requirements.txt安装:
```
pip install -r requirements.txt
```
zhaoying1's avatar
zhaoying1 committed
63

zhaoying1's avatar
zhaoying1 committed
64
65
66
67
68
69
70
71
72
## 数据集
本仓库以 [ADGEN](https://aclanthology.org/D19-1321.pdf) (广告生成) 数据集为例介绍代码的使用方法,该数据集任务为根据输入(content)生成一段广告词(summary),以下为下载地址:
- [Google Drive](https://drive.google.com/file/d/13_vf0xRTQsyneRKdD1bZIr93vBGOczrk/view?usp=sharing) 或者 [Tsinghua Cloud](https://cloud.tsinghua.edu.cn/f/b3f119a008264b1cabd1/?dl=1)
下载处理好的 ADGEN 数据集,将解压后的AdvertiseGen目录放到 [ptuning](./ptuning)本目录下。数据集目录结构如下:
```
 ── AdvertiseGen
    │   ├── dev.json
    │   └── train.json
```
zhaoying1's avatar
zhaoying1 committed
73
74
75
## 模型下载
Hugging Face模型下载地址:
[ChatGLM-6B](https://huggingface.co/THUDM/chatglm-6b)
zhaoying1's avatar
zhaoying1 committed
76

zhaoying1's avatar
zhaoying1 committed
77
78
79
## 训练

### P-tuning v2 微调训练
zhaoying1's avatar
zhaoying1 committed
80
本仓库实现了对于ChatGLM-6B模型基于[P-Tuning v2](https://github.com/THUDM/P-tuning-v2)的微调。P-Tuning v2是由清华大学提出的一种高效参数微调方法。
zhaoying1's avatar
zhaoying1 committed
81

zhaoying1's avatar
zhaoying1 committed
82
#### 单机多卡训练
zhaoying1's avatar
zhaoying1 committed
83
84
85
86
87
```
    cd ptuning
    bash ptuning_train.sh
```
注意:请根据自己的需求配置其中的模型路径、数据集路径、batchsize、学习率等参数;
zhaoying1's avatar
zhaoying1 committed
88

zhaoying1's avatar
zhaoying1 committed
89
#### 推理测评
zhaoying1's avatar
zhaoying1 committed
90
在 P-tuning v2 训练时模型只保存 PrefixEncoder 部分的参数,所以在推理时需要同时加载原 ChatGLM-6B 模型以及 PrefixEncoder 的权重,可直接运行一下命令:
zhaoying1's avatar
zhaoying1 committed
91
```
zhaoying1's avatar
zhaoying1 committed
92
    cd ptuning
zhaoying1's avatar
zhaoying1 committed
93
94
    bash evaluate_ptuning.sh
```
zhaoying1's avatar
zhaoying1 committed
95
#### Results
zhaoying1's avatar
zhaoying1 committed
96
97
98
99
100
101
- 训练Loss
<div align="center">
<img src="./ptuning/media/6B_ds_pt_bs16_accum1_4cards_zero2_5e-3.jpg" width="400" height="300">
</div>

- 推理测试结果:
zhaoying1's avatar
zhaoying1 committed
102
103
104
105
106

| Checkpoint | Training Loss |BLEU-4 | Rouge-1 |  Rouge-2 | Rouge-l |
| :------: | :------: |:------: | :------: |:------: | :------: |
| 2000 steps |  3.57 | 7.9777 | 31.0344 |  6.981 | 24.7393 |

zhaoying1's avatar
zhaoying1 committed
107
### Finetune全参数微调
zhaoying1's avatar
zhaoying1 committed
108

zhaoying1's avatar
zhaoying1 committed
109
#### 单机多卡训练
zhaoying1's avatar
zhaoying1 committed
110
```
zhaoying1's avatar
zhaoying1 committed
111
112
    cd ptuning
    bash ft_train.sh
zhaoying1's avatar
zhaoying1 committed
113
114
```
注意:请根据自己的需求配置其中的模型路径、数据集路径、batchsize、学习率等参数;
zhaoying1's avatar
zhaoying1 committed
115

zhaoying1's avatar
zhaoying1 committed
116
#### 集群训练
zhaoying1's avatar
zhaoying1 committed
117
```
zhaoying1's avatar
zhaoying1 committed
118
    cd ptuning/multi_node
zhaoying1's avatar
zhaoying1 committed
119
    bash run_train.sh
zhaoying1's avatar
zhaoying1 committed
120
121
```
注意:请根据自己的需求配置其中的模型路径、数据集路径、batchsize、学习率等参数;
zhaoying1's avatar
zhaoying1 committed
122
123


zhaoying1's avatar
zhaoying1 committed
124
#### 推理测评
zhaoying1's avatar
zhaoying1 committed
125
```
zhaoying1's avatar
zhaoying1 committed
126
127
    cd ptuning
    bash evaluate_ft.sh
zhaoying1's avatar
zhaoying1 committed
128
```
zhaoying1's avatar
zhaoying1 committed
129
#### Results
zhaoying1's avatar
zhaoying1 committed
130
131
132
133
134
135
- 训练Loss
<div align="center">
<img src="./ptuning/media/6B_ds_ft_bs32_accum1_4cards_zero3_5e-5.jpg" width="400" height="300">
</div>

- 推理测试结果:
zhaoying1's avatar
zhaoying1 committed
136
137
138
139
140
141
142
143
144
145
146
147
148
149

| Checkpoint | Training Loss |BLEU-4 | Rouge-1 |  Rouge-2 | Rouge-l |
| :------: | :------: |:------: | :------: |:------: | :------: |
| 3000 steps |  2.3398 | 7.6501 | 29.2229 | 6.466 | 23.8506 |


<!-- ## 评估结果
|               | Finetune | P-tuning v2 | LoRA |
| ------------- | ----------- | ----- | ------------- |
| BLEU-4        | 8.01    | 8.10 | 7.62 |
| Rouge-1       | 31.23  | 31.12 | 30.60 |
| Rouge-2       | 7.36    | 7.11 | 6.96 |
| Rouge-l       | 25.08  | 24.97 | 24.80 |
| Training Loss | 3.00 | 3.57 | 3.32 | -->
zhaoying1's avatar
zhaoying1 committed
150
151


zhaoying1's avatar
zhaoying1 committed
152
153
### LoRA 微调训练
#### 单机多卡训练
zhaoying1's avatar
zhaoying1 committed
154
155
156
157
```
    cd ptuning
    bash lora_train.sh
```
zhaoying1's avatar
zhaoying1 committed
158
#### LoRA推理
zhaoying1's avatar
zhaoying1 committed
159
160
161
162
163
164
```
    python infer_lora.py
```



zhaoying1's avatar
zhaoying1 committed
165
## 推理
zhaoying1's avatar
zhaoying1 committed
166
167
168
运行如下命令:

    python cli_demo.py
yuguo960516yuguo's avatar
1.0  
yuguo960516yuguo committed
169

zhaoying1's avatar
zhaoying1 committed
170
171
程序会在命令行中进行交互式的对话,在命令行中输入指示并回车即可生成回复,输入 clear 可以清空对话历史,输入 stop 终止程序。

yuguo960516yuguo's avatar
1.0  
yuguo960516yuguo committed
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
## 重新 pretrain

由于当前的 [GLM-130B](https://github.com/THUDM/GLM-130B#news) 与 ChatGLM 的模型结构非常类似,所以对于有训练 GLM-130B 的用户来说,可以通过修改 ChatGLM 的 config.json 使用堆参数的方式将参数量达到130B。该项目为了满足用户对 ChatGLM 重新 pretrain 的需求,继续添加了 simple-pretrain 目录,旨在提供一种改动最小的 pretrain 示例。pretrain步骤如下:

1. 将 simple-pretrain/ptuning 下的文件移到本 ptuning 目录下,替换相关文件

2. 将 modeling_chatglm.py 移到[ ChatGLM 模型](https://huggingface.co/THUDM/chatglm-6b)所在目录替换原始 modeling_chatglm.py 

3. 在本 ptuning 目录下:

   ```
   bash ds_pretrain.sh
   ```

说明:convert.py 可以将原始的txt数据转换成 chatglm 可用的 json 形式的数据集格式。该示例使用指环王1书籍作为预训练数据集。

### 实验设置

```
LR=1e-5

MASTER_PORT=$(shuf -n 1 -i 10000-65535)

HIP_VISIBLE_DEVICES=0,1,2,3 deepspeed --num_gpus=4 --master_port $MASTER_PORT main.py \
    --deepspeed deepspeed.json \
    --do_train \
    --train_file The-Lord-of-the-Rings-1.json \
    --prompt_column prompt \
    --response_column response \
    --overwrite_cache \
    --model_name_or_path THUDM/chatglm-6b \
    --output_dir ./output/pretrain \
    --overwrite_output_dir \
    --max_source_length 8 \
    --max_target_length 128 \
    --per_device_train_batch_size 16 \
    --per_device_eval_batch_size 4 \
    --gradient_accumulation_steps 4 \
    --predict_with_generate \
    --max_steps 2000 \
    --logging_steps 5 \
    --save_steps 1000 \
    --learning_rate $LR \
    --fp16
```

### 训练loss收敛情况

由于该示例预训练数据集较小,loss会降的至较低水平到0.1左右。
zhaoying1's avatar
zhaoying1 committed
221
222
223
<div align="center">
<img src="./ptuning/media/pretrain.jpeg" width="400" height="300">
</div>
yuguo960516yuguo's avatar
1.0  
yuguo960516yuguo committed
224
225
226
227


## 强化学习(RLHF)微调方案

yuguo960516yuguo's avatar
readme  
yuguo960516yuguo committed
228
目前在 DCU 上 ChatGLM 使用强化学习微调有两种方案可以走通:
yuguo960516yuguo's avatar
1.0  
yuguo960516yuguo committed
229

yuguo960516yuguo's avatar
readme  
yuguo960516yuguo committed
230
- 使用 Lora,只更新低秩适应层,可以直接参考项目:https://github.com/hiyouga/ChatGLM-Efficient-Tuning/blob/main/examples/covid_doctor.md
yuguo960516yuguo's avatar
1.0  
yuguo960516yuguo committed
231
232
- 使用 DeepSpeed-Chat 方案全参微调,目前已经适配完成,欢迎尝试:https://github.com/yuguo-Jack/ChatGLM-6B-in-DeepSpeed-Chat

zhaoying1's avatar
zhaoying1 committed
233
234
235
236
237
238
239
240
241
242
243

## 应用场景

### 算法类别

`自然语言处理`

### 热点应用行业

`nlp,智能聊天助手,科研`

yuguo960516yuguo's avatar
1.0  
yuguo960516yuguo committed
244
245
## 源码仓库及问题反馈

zhaoying1's avatar
zhaoying1 committed
246
- https://developer.hpccube.com/codes/modelzoo/chatglm
yuguo960516yuguo's avatar
1.0  
yuguo960516yuguo committed
247

zhaoying1's avatar
zhaoying1 committed
248
249
## 参考

zhaoying1's avatar
zhaoying1 committed
250
- [THUDM/ChatGLM-6B](https://github.com/THUDM/ChatGLM-6B/tree/main)
zhaoying1's avatar
zhaoying1 committed
251