Unverified Commit 155e2023 authored by binmakeswell's avatar binmakeswell Committed by GitHub
Browse files

[example] update auto_parallel img path (#1910)

parent d5c5bc21
...@@ -29,14 +29,14 @@ quickly deploy large AI model training and inference, reducing large AI model tr ...@@ -29,14 +29,14 @@ quickly deploy large AI model training and inference, reducing large AI model tr
- Try sequence parallelism with BERT - Try sequence parallelism with BERT
- Combination of data/pipeline/sequence parallelism - Combination of data/pipeline/sequence parallelism
- Faster training and longer sequence length - Faster training and longer sequence length
- Large Batch Training Optimization
- Comparison of small/large batch size with SGD/LARS optimizer
- Acceleration from a larger batch size
- Auto-Parallelism - Auto-Parallelism
- Parallelism with normal non-distributed training code - Parallelism with normal non-distributed training code
- Model tracing + solution solving + runtime communication inserting all in one auto-parallelism system - Model tracing + solution solving + runtime communication inserting all in one auto-parallelism system
- Try single program, multiple data (SPMD) parallel with auto-parallelism SPMD solver on ResNet50 - Try single program, multiple data (SPMD) parallel with auto-parallelism SPMD solver on ResNet50
- Large Batch Training Optimization - Fine-tuning and Serving for OPT
- Comparison of small/large batch size with SGD/LARS optimizer
- Acceleration from a larger batch size
- Fine-tuning and Serving for OPT from Hugging Face
- Try OPT model imported from Hugging Face with Colossal-AI - Try OPT model imported from Hugging Face with Colossal-AI
- Fine-tuning OPT with limited hardware using ZeRO, Gemini and parallelism - Fine-tuning OPT with limited hardware using ZeRO, Gemini and parallelism
- Deploy the fine-tuned model to inference service - Deploy the fine-tuned model to inference service
......
...@@ -66,10 +66,10 @@ python demo_gpt2_medium.py ...@@ -66,10 +66,10 @@ python demo_gpt2_medium.py
There are some results for your reference There are some results for your reference
### ResNet 50 ### ResNet 50
![](./imgs/resnet50_benchmark.png) ![](https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/tutorial/resnet50_benchmark.png)
### GPT2 Medium ### GPT2 Medium
![](./imgs/gpt2_benchmark.png) ![](https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/tutorial/gpt2_benchmark.png)
We also prepare the demo `demo_resnet152.py` to manifest the benefit of auto activation with large batch, the usage is listed as follows We also prepare the demo `demo_resnet152.py` to manifest the benefit of auto activation with large batch, the usage is listed as follows
```bash ```bash
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment