Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
JIUTIAN-139MoE-Chat
Commits
0acb55cc
Commit
0acb55cc
authored
Oct 12, 2024
by
ACzhangchao
Browse files
add png
parent
a57fb071
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
2 additions
and
2 deletions
+2
-2
MoE.png
MoE.png
+0
-0
README.md
README.md
+2
-2
jiutian.png
jiutian.png
+0
-0
No files found.
MoE.png
0 → 100644
View file @
0acb55cc
194 KB
README.md
View file @
0acb55cc
...
...
@@ -8,13 +8,13 @@ https://www.modelscope.cn/models/JiuTian-AI/JIUTIAN-139MoE-chat/file/view/master
JIUTIAN-139MoE是一个拥有130亿参数的大型语言模型,使用解码器型的MoE架构,包含一对大专家和六个小专家。模型支持在不同GPU和NPU集群上训练,并能无损切换。在FFN层采用MoE设计,有特殊的激活和路由机制。


## 算法原理
JIUTIAN-139MoE利用Mixture-of-Experts (MoE) 架构,通过不同规模的专家网络处理不同的数据特征,并通过门控机制智能地分配任务给最合适的专家,从而提高模型处理复杂问题的能力。


## 环境配置
...
...
jiutian.png
0 → 100644
View file @
0acb55cc
166 KB
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment