Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
ColossalAI
Commits
deaf99f4
Unverified
Commit
deaf99f4
authored
Apr 14, 2022
by
binmakeswell
Committed by
GitHub
Apr 14, 2022
Browse files
[readme] sync CN readme (#766)
parent
6e553748
Changes
2
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
17 additions
and
13 deletions
+17
-13
README-zh-Hans.md
README-zh-Hans.md
+14
-10
README.md
README.md
+3
-3
No files found.
README-zh-Hans.md
View file @
deaf99f4
...
...
@@ -71,26 +71,30 @@
Colossal-AI 为您提供了一系列并行训练组件。我们的目标是让您的分布式 AI 模型训练像普通的单 GPU 模型一样简单。我们提供的友好工具可以让您在几行代码内快速开始分布式训练。
-
数据并行
-
流水线并行
-
1维, 2维, 2.5维, 3维张量并行
-
序列并行
-
友好的 trainer 和 engine
-
可扩展新的并行方式
-
混合精度
-
零冗余优化器 (ZeRO)
-
并行化策略
-
数据并行
-
流水线并行
-
1维,
[
2维
](
https://arxiv.org/abs/2104.05343
)
,
[
2.5维
](
https://arxiv.org/abs/2105.14500
)
,
[
3维
](
https://arxiv.org/abs/2105.14450
)
张量并行
-
[
序列并行
](
https://arxiv.org/abs/2105.13120
)
-
[
零冗余优化器 (ZeRO)
](
https://arxiv.org/abs/2108.05818
)
-
异构内存管理
-
[
PatrickStar
](
https://arxiv.org/abs/2108.05818
)
-
使用友好
-
基于参数文件的并行化
<p
align=
"right"
>
(
<a
href=
"#top"
>
返回顶端
</a>
)
</p>
## 展示样例
### ViT
<p
align=
"center"
>
<img
src=
"https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/ViT.png"
width=
"450"
/>
</p>
-
14倍批大小和5倍训练速度(张量并行=64)
### GPT-3
<p
align=
"center"
>
<img
src=
"https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/GPT3.png"
width=
700/
>
</p>
-
释放 50% GPU 资源占用, 或 10.7% 加速
...
...
README.md
View file @
deaf99f4
...
...
@@ -76,15 +76,15 @@ distributed training in a few lines.
-
Parallelism strategies
-
Data Parallelism
-
Pipeline Parallelism
-
1D,
[
2D
](
https://arxiv.org/abs/2104.05343
)
,
[
2.5D
](
https://arxiv.org/abs/2105.14500
)
, 3D Tensor
p
arallelism
-
[
Sequence
p
arallelism
](
https://arxiv.org/abs/2105.13120
)
-
1D,
[
2D
](
https://arxiv.org/abs/2104.05343
)
,
[
2.5D
](
https://arxiv.org/abs/2105.14500
)
,
[
3D
](
https://arxiv.org/abs/2105.14450
)
Tensor
P
arallelism
-
[
Sequence
P
arallelism
](
https://arxiv.org/abs/2105.13120
)
-
[
Zero Redundancy Optimizer (ZeRO)
](
https://arxiv.org/abs/2108.05818
)
-
Heterogeneous Memory Menagement
-
[
PatrickStar
](
https://arxiv.org/abs/2108.05818
)
-
Friendly Usage
-
C
onfiguration file
based
-
Parallelism based on c
onfiguration file
<p
align=
"right"
>
(
<a
href=
"#top"
>
back to top
</a>
)
</p>
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment