Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
ColossalAI
Commits
3312d716
Commit
3312d716
authored
Feb 28, 2022
by
Sze-qq
Committed by
Frank Lee
Mar 11, 2022
Browse files
update experimental visualization (#253)
parent
753035ed
Changes
7
Hide whitespace changes
Inline
Side-by-side
Showing
7 changed files
with
20 additions
and
8 deletions
+20
-8
README-zh-Hans.md
README-zh-Hans.md
+10
-4
README.md
README.md
+10
-4
docs/images/GPT3Table.png
docs/images/GPT3Table.png
+0
-0
docs/images/allinone/BERT_allin1.png
docs/images/allinone/BERT_allin1.png
+0
-0
docs/images/allinone/GPT2_allin1.png
docs/images/allinone/GPT2_allin1.png
+0
-0
docs/images/allinone/GPT3_allin1.png
docs/images/allinone/GPT3_allin1.png
+0
-0
docs/images/update/vit.png
docs/images/update/vit.png
+0
-0
No files found.
README-zh-Hans.md
View file @
3312d716
...
...
@@ -34,21 +34,27 @@ Colossal-AI为您提供了一系列并行训练组件。我们的目标是让您
## 样例
### ViT
<img
src=
"./docs/images/ViT_TP.png"
width=
"400"
/>
<img
src=
"./docs/images/update/vit.png"
width=
"450"
/>
-
14倍批大小
-
5倍训练速度
### GPT-3 & GPT-2
### GPT-3
<img
src=
"./docs/images/allinone/GPT3_allin1.png"
width=
700/
>

-
GPT-3:释放 50% GPU 资源占用, 或 10.7% 加速
### GPT-2
<img
src=
"./docs/images/allinone/GPT2_allin1.png"
width=
800/
>
-
GPT-2:降低11倍GPU显存占用,或超线性扩展
### BERT
<img
src=
"./docs/images/allinone/BERT_allin1.png"
width=
800/
>

-
2倍训练速度
-
1.5倍序列长度
...
...
README.md
View file @
3312d716
...
...
@@ -37,21 +37,27 @@ distributed training in a few lines.
## Examples
### ViT
<img
src=
"./docs/images/
ViT_TP
.png"
width=
"4
0
0"
/>
<img
src=
"./docs/images/
update/vit
.png"
width=
"4
5
0"
/>
-
14x larger batch size
-
5x faster training
### GPT-3
& GPT-2
### GPT-3

<img
src=
"
./docs/images/
allinone/GPT3_allin1.png"
width=
700/
>
-
Free 50% GPU resources, or 10.7% acceleration for GPT-3
### GPT-2
<img
src=
"./docs/images/allinone/GPT2_allin1.png"
width=
800/
>
-
11x lower GPU RAM, or superlinear scaling for GPT-2
### BERT
<img
src=
"./docs/images/allinone/BERT_allin1.png"
width=
800/
>

-
2x faster training
-
50% longer sequence length
...
...
docs/images/GPT3Table.png
0 → 100644
View file @
3312d716
263 KB
docs/images/allinone/BERT_allin1.png
0 → 100644
View file @
3312d716
1.07 MB
docs/images/allinone/GPT2_allin1.png
0 → 100644
View file @
3312d716
603 KB
docs/images/allinone/GPT3_allin1.png
0 → 100644
View file @
3312d716
1.32 MB
docs/images/update/vit.png
0 → 100644
View file @
3312d716
545 KB
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment