Unverified Commit a255a38f authored by Jiatong (Julius) Han's avatar Jiatong (Julius) Han Committed by GitHub
Browse files

[example] Polish README.md (#2658)

* [tutorial] polish readme.md

* [example] Update README.md
parent cd4f02be
...@@ -54,13 +54,13 @@ However, it requires some efforts to start if facing a new model structure. ...@@ -54,13 +54,13 @@ However, it requires some efforts to start if facing a new model structure.
bash run_gemini.sh bash run_gemini.sh
``` ```
The `train_gpt_demo.py` provides three distributed plans, you can choose the plan you want in `run_gemini.sh`. The Colossal-AI leverages Tensor Parallel and Gemini + ZeRO DDP. The `train_gpt_demo.py` provides three distributed plans (except ones already provided by PyTorch), you can choose the plan you want in `run_gemini.sh`. The CAI_Gemini leverages Tensor Parallel and Gemini + ZeRO DDP. For their differences, you may check out the answer to issue [here](https://github.com/hpcaitech/ColossalAI/issues/2590#issuecomment-1418766581).
- Colossal-AI - ZeRO1 (CAI_ZeRO1)
- ZeRO1 (Colossal-AI) - ZeRO2 (CAI_ZeRO2)
- ZeRO2 (Colossal-AI) - Gemini + ZeRO DDP (CAI_Gemini)
- Pytorch DDP - Pytorch DDP (Pytorch_DDP)
- Pytorch ZeRO - Pytorch ZeRO (Pytorch_ZeRO)
### Titans (Tensor Parallelism) + ZeRO + Pipeline Parallelism ### Titans (Tensor Parallelism) + ZeRO + Pipeline Parallelism
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment