Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
ChatGLM-6B_pytorch
Commits
f51eee89
Commit
f51eee89
authored
Sep 21, 2023
by
zhaoying1
Browse files
调整为标准格式
parent
eed209e7
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
2 additions
and
4 deletions
+2
-4
README.md
README.md
+2
-4
ptuning/multi_node/run_train.sh
ptuning/multi_node/run_train.sh
+0
-0
ptuning/multi_node/run_train_single.sh
ptuning/multi_node/run_train_single.sh
+0
-0
No files found.
README.md
View file @
f51eee89
...
...
@@ -6,9 +6,7 @@
## 模型结构
ChatGLM-6B 是清华大学开源的开源的、支持中英双语的对话语言模型,基于
[
General Language Model (GLM)
](
https://github.com/THUDM/GLM
)
架构,具有 62 亿参数。ChatGLM-6B 使用了和 ChatGPT 相似的技术,针对中文问答和对话进行了优化。经过约 1T 标识符的中英双语训练,辅以监督微调、反馈自助、人类反馈强化学习等技术的加持,62 亿参数的 ChatGLM-6B 已经能生成相当符合人类偏好的回答。
<div
align=
"center"
>
<img
src=
"ptuning/media/GLMblock.png"
width=
"550"
height=
"200"
>
</div>
以下是ChatGLM-6B的主要网络参数配置:
...
...
@@ -117,7 +115,7 @@ Hugging Face模型下载地址:
#### 集群训练
```
cd ptuning/
slurm_scripts
cd ptuning/
multi_node
bash run_train.sh
```
注意:请根据自己的需求配置其中的模型路径、数据集路径、batchsize、学习率等参数;
...
...
ptuning/multi_node
l
/run_train.sh
→
ptuning/multi_node/run_train.sh
View file @
f51eee89
File moved
ptuning/multi_node
l
/run_train_single.sh
→
ptuning/multi_node/run_train_single.sh
View file @
f51eee89
File moved
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment