Skip to content

GitLab

Commit 1c88dd43 authored by Shen Chenhui's avatar Shen Chenhui Committed by Frank Lee
Browse files

Fix/format (#366)

parent 0db43fa9
Show whitespace changes
Inline Side-by-side
Showing with 4 additions and 6 deletions
README.md
View file @ 1c88dd43
...@@ -38,24 +38,22 @@ distributed training in a few lines. ...@@ -38,24 +38,22 @@ distributed training in a few lines.
## Examples ## Examples
### ViT ### ViT
<img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/ViT.png" width="450" />
<img src="./docs/images/ViT.png" width="450" />
- 14x larger batch size, and 5x faster training for Tensor Parallel = 64 - 14x larger batch size, and 5x faster training for Tensor Parallel = 64
### GPT-3 ### GPT-3
<img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/GPT3.png" width=700/>
<img src="./docs/images/GPT3.png" width=700/>
- Free 50% GPU resources, or 10.7% acceleration - Free 50% GPU resources, or 10.7% acceleration
### GPT-2 ### GPT-2
<img src="./docs/images/GPT2.png" width=800/> <img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/GPT2.png" width=800/>
- 11x lower GPU RAM, or superlinear scaling - 11x lower GPU RAM, or superlinear scaling
### BERT ### BERT
<img src="./docs/images/BERT.png" width=800/> <img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/BERT.png" width=800/>
- 2x faster training, or 50% longer sequence length - 2x faster training, or 50% longer sequence length
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment