[Docs]Add deploy docs (#1373)

* add deploy docs

* fix docs

* format docs

* fix docs and readme

* fix readme

* fix docs

* fix docs

* fix lint tag

* fix docs

README.md

@@ -35,6 +35,8 @@ In the [nuScenes 3D detection challenge](https://www.nuscenes.org/object-detecti
 
 Code and models for the best vision-only method, [FCOS3D](https://arxiv.org/abs/2104.10956), have been released. Please stay tuned for [MoCa](https://arxiv.org/abs/2012.12741).
 
+MMDeploy has supported some MMDetection3d model deployment.
+
 Documentation: https://mmdetection3d.readthedocs.io/
 
 ## Introduction
@@ -228,7 +230,6 @@ Results and models are available in the [model zoo](docs/en/model_zoo.md).
 | PGD                  | ✓        | ☐        | ☐        | ✗         | ✗         | ☐     | ☐        | ☐     | ✗
 | MonoFlex             | ✗        | ✗        | ✗        | ✗         | ✗         | ✗     | ✗        | ✗     | ✓
 
-
 **Note:** All the about **300+ models, methods of 40+ papers** in 2D detection supported by [MMDetection](https://github.com/open-mmlab/mmdetection/blob/master/docs/en/model_zoo.md) can be trained or used in this codebase.
 
 ## Installation
@@ -241,6 +242,9 @@ Please see [getting_started.md](docs/en/getting_started.md) for the basic usage
 
 Please refer to [FAQ](docs/en/faq.md) for frequently asked questions. When updating the version of MMDetection3D, please also check the [compatibility doc](docs/en/compatibility.md) to be aware of the BC-breaking updates introduced in each version.
 
+## Model deployment
+
+Now MMDeploy has supported some MMDetection3D model deployment. Please refer to [model_deployment.md](docs/en/tutorials/model_deployment.md) for more details.
 ## Citation
 
 If you find this project useful in your research, please consider cite:

README_zh-CN.md

@@ -35,6 +35,8 @@
 
 最好的纯视觉方法 [FCOS3D](https://arxiv.org/abs/2104.10956) 的代码和模型已经发布。请继续关注我们的多模态检测器 [MoCa](https://arxiv.org/abs/2012.12741)。
 
+MMDeploy 已经支持了部分 MMDetection3D 模型的部署。
+
 文档: https://mmdetection3d.readthedocs.io/
 
 ## 简介
@@ -241,6 +243,10 @@ MMDetection3D 是一个基于 PyTorch 的目标检测开源工具箱, 下一代
 
 请参考 [FAQ](docs/zh_cn/faq.md) 查看一些常见的问题与解答。在升级 MMDetection3D 的版本时，请查看[兼容性文档](docs/zh_cn/compatibility.md)以知晓每个版本引入的不与之前版本兼容的更新。
 
+## 模型部署
+
+现在 MMDeploy 已经支持了一些 MMDetection3D 模型的部署。请参考 [model_deployment.md](docs/zh_cn/tutorials/model_deployment.md)了解更多细节。
+
 ## 引用
 
 如果你觉得本项目对你的研究工作有所帮助，请参考如下 bibtex 引用 MMdetection3D

docs/en/tutorials/model_deployment.md

+# Tutorial 7: MMDetection3D model deployment
+
+To meet the speed requirement of the model in practical use, usually, we deploy the trained model to inference backends. [MMDeploy](https://github.com/open-mmlab/mmdeploy) is OpenMMLab model deployment framework. Now MMDeploy has supported MMDetection3D model deployment, and you can deploy the trained model to inference backends by MMDeploy.
+
+## Prerequisite
+
+### Install MMDeploy
+
+```bash
+git clone -b master git@github.com:open-mmlab/mmdeploy.git
+cd mmdeploy
+git submodule update --init --recursive
+```
+
+### Install backend and build custom ops
+
+According to MMDeploy documentation, choose to install the inference backend and build custom ops. Now supported inference backends for MMDetection3D include [OnnxRuntime](https://mmdeploy.readthedocs.io/en/latest/backends/onnxruntime.html), [TensorRT](https://mmdeploy.readthedocs.io/en/latest/backends/tensorrt.html), [OpenVINO](https://mmdeploy.readthedocs.io/en/latest/backends/openvino.html).
+
+## Export model
+
+Export the Pytorch model of MMDetection3D to the ONNX model file and the model file required by the backend. You could refer to MMDeploy docs [how to convert model](https://mmdeploy.readthedocs.io/en/latest/tutorials/how_to_convert_model.html).
+
+```bash
+python ./tools/deploy.py \
+    ${DEPLOY_CFG_PATH} \
+    ${MODEL_CFG_PATH} \
+    ${MODEL_CHECKPOINT_PATH} \
+    ${INPUT_IMG} \
+    --test-img ${TEST_IMG} \
+    --work-dir ${WORK_DIR} \
+    --calib-dataset-cfg ${CALIB_DATA_CFG} \
+    --device ${DEVICE} \
+    --log-level INFO \
+    --show \
+    --dump-info
+```
+
+### Description of all arguments
+
+* `deploy_cfg` : The path of deploy config file in MMDeploy codebase.
+* `model_cfg` : The path of model config file in OpenMMLab codebase.
+* `checkpoint` : The path of model checkpoint file.
+* `img` : The path of point cloud file or image file that used to convert model.
+* `--test-img` : The path of image file that used to test model. If not specified, it will be set to `None`.
+* `--work-dir` : The path of work directory that used to save logs and models.
+* `--calib-dataset-cfg` : Only valid in int8 mode. Config used for calibration. If not specified, it will be set to `None` and  use "val" dataset in model config for calibration.
+* `--device` : The device used for conversion. If not specified, it will be set to `cpu`.
+* `--log-level` : To set log level which in `'CRITICAL', 'FATAL', 'ERROR', 'WARN', 'WARNING', 'INFO', 'DEBUG', 'NOTSET'`. If not specified, it will be set to `INFO`.
+* `--show` : Whether to show detection outputs.
+* `--dump-info` : Whether to output information for SDK.
+
+### Example
+
+```bash
+cd mmdeploy \
+python tools/deploy.py \
+configs/mmdet3d/voxel-detection/voxel-detection_tensorrt_dynamic-kitti.py \
+${$MMDET3D_DIR}/configs/pointpillars/hv_pointpillars_secfpn_6x8_160e_kitti-3d-3class.py \
+${$MMDET3D_DIR}/checkpoints/hv_pointpillars_secfpn_6x8_160e_kitti-3d-3class_20200620_230421-aa0f3adb.pth \
+${$MMDET3D_DIR}/demo/data/kitti/kitti_000008.bin \
+--work-dir
+work-dir \
+--device
+cuda:0 \
+--show
+```
+
+## Inference Model
+
+Now you can do model inference with the APIs provided by the backend. But what if you want to test the model instantly? We have some backend wrappers for you.
+
+```python
+from mmdeploy.apis import inference_model
+
+result = inference_model(model_cfg, deploy_cfg, backend_files, img=img, device=device)
+```
+
+The `inference_model` will create a wrapper module and do the inference for you. The result has the same format as the original OpenMMLab repo.
+
+## Evaluate model (Optional)
+
+You can test the accuracy and speed of the model in the inference backend. You could refer to MMDeploy docs [how to measure performance of models](https://mmdeploy.readthedocs.io/en/latest/tutorials/how_to_measure_performance_of_models.html).
+
+```bash
+python tools/test.py \
+${DEPLOY_CFG} \
+${MODEL_CFG} \
+--model ${BACKEND_MODEL_FILES} \
+[--out ${OUTPUT_PKL_FILE}] \
+[--format-only] \
+[--metrics ${METRICS}] \
+[--show] \
+[--show-dir ${OUTPUT_IMAGE_DIR}] \
+[--show-score-thr ${SHOW_SCORE_THR}] \
+--device ${DEVICE} \
+[--cfg-options ${CFG_OPTIONS}] \
+[--metric-options ${METRIC_OPTIONS}]
+[--log2file work_dirs/output.txt]
+```
+
+### Example
+
+```bash
+cd mmdeploy \
+python tools/test.py \
+configs/mmdet3d/voxel-detection/voxel-detection_onnxruntime_dynamic.py \
+${MMDET3D_DIR}/configs/centerpoint/centerpoint_02pillar_second_secfpn_circlenms_4x8_cyclic_20e_nus.py \
+--model
+work-dir/end2end.onnx \
+--metrics
+bbox \
+--device
+cpu
+```
+
+## Supported models
+
+| Model                | TorchScript | OnnxRuntime | TensorRT | NCNN  | PPLNN | OpenVINO | Model config                                                                           |
+| -------------------- | :---------: | :---------: | :------: | :---: | :---: | :------: | -------------------------------------------------------------------------------------- |
+| PointPillars         |      ?      |      Y      |    Y     |   N   |   N   |    Y     | [config](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/pointpillars) |
+| CenterPoint (pillar) |      ?      |      Y      |    Y     |   N   |   N   |    Y     | [config](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/centerpoint)  |
+
+## Note
+
+* MMDeploy version >= 0.4.0.
+* Currently, CenterPoint has only supported the pillar version.

docs/zh_cn/tutorials/model_deployment.md

+# 教程 7: MMDet3D模型部署
+
+为了满足在实际使用过程中遇到的算法模型的速度需求，通常我们会将训练好的模型部署到各种推理后端上。 [MMDeploy](https://github.com/open-mmlab/mmdeploy) 是 OpenMMLab 系列算法库的部署框架，现在 MMDeploy 已经支持了MMDetection3D，我们可以通过 MMDeploy 将训练好的模型部署到各种推理后端上。
+
+## 准备
+
+### 安装MMDeploy
+
+```bash
+git clone -b master git@github.com:open-mmlab/mmdeploy.git
+cd mmdeploy
+git submodule update --init --recursive
+```
+
+### 安装推理后端编译自定义算子
+
+根据 MMDeploy 的文档选择安装推理后端并编译自定义算子，目前 MMDet3D 模型支持了的推理后端有 [OnnxRuntime](https://mmdeploy.readthedocs.io/en/latest/backends/onnxruntime.html), [TensorRT](https://mmdeploy.readthedocs.io/en/latest/backends/tensorrt.html), [OpenVINO](https://mmdeploy.readthedocs.io/en/latest/backends/openvino.html) 。
+
+## 模型导出
+
+将 MMDet3D 训练好的 Pytorch 模型转换成 ONNX 模型文件和推理后端所需要的模型文件。你可以参考 MMDeploy 的文档 [how_to_convert_model.md](https://github.com/open-mmlab/mmdeploy/blob/master/docs/zh_cn/tutorials/how_to_convert_model.md)。
+
+```bash
+python ./tools/deploy.py \
+    ${DEPLOY_CFG_PATH} \
+    ${MODEL_CFG_PATH} \
+    ${MODEL_CHECKPOINT_PATH} \
+    ${INPUT_IMG} \
+    --test-img ${TEST_IMG} \
+    --work-dir ${WORK_DIR} \
+    --calib-dataset-cfg ${CALIB_DATA_CFG} \
+    --device ${DEVICE} \
+    --log-level INFO \
+    --show \
+    --dump-info
+```
+
+### 参数描述
+
+* deploy_cfg : MMDeploy 中用于部署的配置文件路径。
+* model_cfg : OpenMMLab 系列代码库中使用的模型配置文件路径。
+* checkpoint : OpenMMLab 系列代码库的模型文件路径。
+* img : 用于模型转换时使用的点云文件或图像文件路径。
+* --test-img : 用于测试模型的图像文件路径。默认设置成None。
+* --work-dir : 工作目录，用来保存日志和模型文件。
+* --calib-dataset-cfg : 此参数只有int8模式下生效，用于校准
+* 数据集配置文件。若在int8模式下未传入参数，则会自动使用模型配置文件中的'val'数据集进行校准。
+* --device : 用于模型转换的设备。 默认是cpu。
+* --log-level : 设置日记的等级，选项包括'CRITICAL'， 'FATAL'， 'ERROR'， 'WARN'， 'WARNING'， 'INFO'， 'DEBUG'， 'NOTSET'。 默认是INFO。
+* --show : 是否显示检测的结果。
+* --dump-info : 是否输出 SDK 信息。
+
+### 示例
+
+```bash
+cd mmdeploy \
+python tools/deploy.py \
+configs/mmdet3d/voxel-detection/voxel-detection_tensorrt_dynamic-kitti.py \
+${$MMDET3D_DIR}/configs/pointpillars/hv_pointpillars_secfpn_6x8_160e_kitti-3d-3class.py \
+${$MMDET3D_DIR}/checkpoints/hv_pointpillars_secfpn_6x8_160e_kitti-3d-3class_20200620_230421-aa0f3adb.pth \
+${$MMDET3D_DIR}/demo/data/kitti/kitti_000008.bin \
+--work-dir
+work-dir \
+--device
+cuda:0 \
+--show
+```
+
+## 模型推理
+
+现在你可以使用推理后端提供的 API 进行模型推理。但是，如果你想立即测试模型怎么办？ 我们为您准备了一些推理后端的封装。
+
+```python
+from mmdeploy.apis import inference_model
+
+result = inference_model(model_cfg, deploy_cfg, backend_files, img=img, device=device)
+```
+
+`inference_model` 将创建一个推理后端的模块并为你进行推理。推理结果与模型的 OpenMMLab 代码库具有相同的格式。
+
+## 测试模型 (可选)
+
+可以测试部署在推理后端上的模型的精度和速度。你可以参考 [how to measure performance of models](https://mmdeploy.readthedocs.io/en/latest/tutorials/how_to_measure_performance_of_models.html)。
+
+```bash
+python tools/test.py \
+${DEPLOY_CFG} \
+${MODEL_CFG} \
+--model ${BACKEND_MODEL_FILES} \
+[--out ${OUTPUT_PKL_FILE}] \
+[--format-only] \
+[--metrics ${METRICS}] \
+[--show] \
+[--show-dir ${OUTPUT_IMAGE_DIR}] \
+[--show-score-thr ${SHOW_SCORE_THR}] \
+--device ${DEVICE} \
+[--cfg-options ${CFG_OPTIONS}] \
+[--metric-options ${METRIC_OPTIONS}]
+[--log2file work_dirs/output.txt]
+```
+
+### 示例
+
+```bash
+cd mmdeploy \
+python tools/test.py \
+configs/mmdet3d/voxel-detection/voxel-detection_onnxruntime_dynamic.py \
+${MMDET3D_DIR}/configs/centerpoint/centerpoint_02pillar_second_secfpn_circlenms_4x8_cyclic_20e_nus.py \
+--model
+work-dir/end2end.onnx \
+--metrics
+bbox \
+--device
+cpu
+```
+
+## 支持模型列表
+
+| Model                | TorchScript | OnnxRuntime | TensorRT | NCNN  | PPLNN | OpenVINO | Model config                                                                           |
+| -------------------- | :---------: | :---------: | :------: | :---: | :---: | :------: | -------------------------------------------------------------------------------------- |
+| PointPillars         |      ?      |      Y      |    Y     |   N   |   N   |    Y     | [config](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/pointpillars) |
+| CenterPoint (pillar) |      ?      |      Y      |    Y     |   N   |   N   |    Y     | [config](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/centerpoint)  |
+
+## 注意
+
+* MMDeploy 的版本需要 >= 0.4.0。
+* 目前 CenterPoint 仅支持了 pillar 版本的。