[doc] update amp document

docs/sidebars.json

@@ -26,11 +26,8 @@
       "collapsed": true,
       "items": [
         "basics/command_line_tool",
-        "basics/launch_colossalai",
-        "basics/booster_api",
-        "basics/booster_plugins",
-        "basics/booster_checkpoint",
         "basics/define_your_config",
+        "basics/launch_colossalai",
         "basics/initialize_features",
         "basics/engine_trainer",
         "basics/configure_parallelization",
@@ -43,10 +40,9 @@
       "label": "Features",
       "collapsed": true,
       "items": [
+        "features/mixed_precision_training_with_booster",
         "features/mixed_precision_training",
-        "features/gradient_accumulation_with_booster",
         "features/gradient_accumulation",
-        "features/gradient_clipping_with_booster",
         "features/gradient_clipping",
         "features/gradient_handler",
         "features/zero_with_chunk",
@@ -62,8 +58,7 @@
           ]
         },
         "features/pipeline_parallel",
-        "features/nvme_offload",
-        "features/cluster_utils"
+        "features/nvme_offload"
       ]
     },
     {

docs/source/en/basics/define_your_config.md

@@ -2,9 +2,6 @@
 
 Author: Guangyang Lu, Shenggui Li, Siqi Mai
 
-> ⚠️ The information on this page is outdated and will be deprecated. Please check [Booster API](../basics/booster_api.md) for more information.
-
-
 **Prerequisite:**
 - [Distributed Training](../concepts/distributed_training.md)
 - [Colossal-AI Overview](../concepts/colossalai_overview.md)
@@ -24,8 +21,7 @@ In this tutorial, we will cover how to define your configuration file.
 ## Configuration Definition
 
 In a configuration file, there are two types of variables. One serves as feature specification and the other serves
-as hyper-parameters. All feature-related variables are reserved keywords. For example, if you want to use mixed precision
-training, you need to use the variable name `fp16` in the config file and follow a pre-defined format.
+as hyper-parameters. All feature-related variables are reserved keywords. For example, if you want to use 1D tensor parallelism, you need to use the variable name `parallel` in the config file and follow a pre-defined format.
 
 ### Feature Specification
 
@@ -37,14 +33,13 @@ To illustrate the use of config file, we use mixed precision training as an exam
 follow the steps below.
 
 1. create a configuration file (e.g. `config.py`, the file name can be anything)
-2. define the mixed precision configuration in the config file. For example, in order to use mixed precision training
-natively provided by PyTorch, you can just write these lines of code below into your config file.
+2. define the hybrid parallelism configuration in the config file. For example, in order to use 1D tensor parallel, you can just write these lines of code below into your config file.
 
    ```python
-   from colossalai.amp import AMP_TYPE
-
-   fp16 = dict(
-     mode=AMP_TYPE.TORCH
+   parallel = dict(
+      data=1,
+      pipeline=1,
+      tensor=dict(size=2, mode='1d'),
    )
    ```
 
@@ -57,7 +52,7 @@ the current directory.
    colossalai.launch(config='./config.py', ...)
    ```
 
-In this way, Colossal-AI knows what features you want to use and will inject this feature during `colossalai.initialize`.
+In this way, Colossal-AI knows what features you want to use and will inject this feature.
 
 ### Global Hyper-parameters
 
@@ -83,3 +78,4 @@ colossalai.launch(config='./config.py', ...)
 print(gpc.config.BATCH_SIZE)
 
 ```
+<!-- doc-test-command: echo "define_your_config.md does not need test" -->

docs/source/en/features/mixed_precision_training.md

-# Auto Mixed Precision Training
+# Auto Mixed Precision Training (Outdated)
 
 Author: Chuanrui Wang, Shenggui Li, Yongbin Li
 
@@ -365,3 +365,4 @@ Use the following command to start the training scripts. You can change `--nproc
 ```python
 python -m torch.distributed.launch --nproc_per_node 4 --master_addr localhost --master_port 29500 train_with_engine.py --config config/config_AMP_torch.py
 ```
+<!-- doc-test-command: torchrun --standalone --nproc_per_node=1 mixed_precision_training.py  -->

docs/source/zh-Hans/basics/define_your_config.md

@@ -2,8 +2,6 @@
 
 作者: Guangyang Lu, Shenggui Li, Siqi Mai
 
-> ⚠️ 此页面上的信息已经过时并将被废弃。请在[Booster API](../basics/booster_api.md)页面查阅更新。
-
 **预备知识:**
 - [分布式训练](../concepts/distributed_training.md)
 - [Colossal-AI 总览](../concepts/colossalai_overview.md)
@@ -20,7 +18,7 @@
 
 ## 配置定义
 
-在一个配置文件中，有两种类型的变量。一种是作为特征说明，另一种是作为超参数。所有与特征相关的变量都是保留关键字。例如，如果您想使用混合精度训练，需要在 config 文件中使用变量名`fp16`，并遵循预先定义的格式。
+在一个配置文件中，有两种类型的变量。一种是作为特征说明，另一种是作为超参数。所有与特征相关的变量都是保留关键字。例如，如果您想使用`1D`张量并行，需要在 config 文件中使用变量名`fp16`，并遵循预先定义的格式。
 
 ### 功能配置
 
@@ -29,13 +27,13 @@ Colossal-AI 提供了一系列的功能来加快训练速度。每个功能都
 为了说明配置文件的使用，我们在这里使用混合精度训练作为例子。您需要遵循以下步骤。
 
 1. 创建一个配置文件（例如 `config.py`，您可以指定任意的文件名）。
-2. 在配置文件中定义混合精度的配置。例如，为了使用 PyTorch 提供的原始混合精度训练，您只需将下面这几行代码写入您的配置文件中。
+2. 在配置文件中定义混合并行的配置。例如，为了使用`1D`张量并行，您只需将下面这几行代码写入您的配置文件中。
 
     ```python
-   from colossalai.amp import AMP_TYPE
-
-   fp16 = dict(
-     mode=AMP_TYPE.TORCH
+   parallel = dict(
+      data=1,
+      pipeline=1,
+      tensor=dict(size=2, mode='1d'),
    )
    ```
 
@@ -47,7 +45,7 @@ Colossal-AI 提供了一系列的功能来加快训练速度。每个功能都
    colossalai.launch(config='./config.py', ...)
    ```
 
-这样，Colossal-AI 便知道您想使用什么功能，并会在 `colossalai.initialize` 期间注入您所需要的功能。
+这样，Colossal-AI 便知道您想使用什么功能，并注入您所需要的功能。
 
 ### 全局超参数
 
@@ -71,3 +69,4 @@ colossalai.launch(config='./config.py', ...)
 print(gpc.config.BATCH_SIZE)
 
 ```
+<!-- doc-test-command: echo "define_your_config.md does not need test" -->

docs/source/zh-Hans/features/mixed_precision_training.md

-# 自动混合精度训练 (AMP)
+# 自动混合精度训练 (旧版本)
 
 作者: Chuanrui Wang, Shenggui Li, Yongbin Li
 
@@ -342,3 +342,4 @@ for epoch in range(gpc.config.NUM_EPOCHS):
 ```python
 python -m torch.distributed.launch --nproc_per_node 4 --master_addr localhost --master_port 29500 train_with_engine.py --config config/config_AMP_torch.py
 ```
+<!-- doc-test-command: torchrun --standalone --nproc_per_node=1 mixed_precision_training.py  -->