Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
zhougaofeng
Yi-1.5-6B-chat
Commits
d4bf3227
Commit
d4bf3227
authored
Nov 07, 2024
by
zhougaofeng
Browse files
Update README.md
parent
21e40ce3
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
1 deletion
+2
-1
README.md
README.md
+2
-1
No files found.
README.md
View file @
d4bf3227
...
...
@@ -7,7 +7,8 @@
-
[https://arxiv.org/abs/2403.04652]
## 算法原理
Yi-1.5-6B 是一个 decoder-only 的 transformer 模型,使用 SwiGLU激活函数、GQA、RoPE等
Yi-1.5是一个 decoder-only 的 transformer 模型,使用 SwiGLU激活函数、GQA、RoPE等是Yi的升级版本,它在Yi的基础上进行了持续预训练,使用了500B(即500十亿)个高质量语料库的token,并且在300万个多样化的微调样本上进行了微调。
与Yi相比,Yi-1.5在编程、数学、推理和指令遵循能力方面表现更强,同时仍然保持了在语言理解、常识推理和阅读理解方面的卓越能力。
## 环境配置
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment