We benchmark Part-A2 with that in [OpenPCDet](https://github.com/open-mmlab/OpenPCDet/tree/b32fbddbe06183507bad433ed99b407cbc2175c2). We report the AP on moderate condition following the KITTI benchmark
We benchmark Part-A2 with that in [OpenPCDet](https://github.com/open-mmlab/OpenPCDet/tree/b32fbddbe06183507bad433ed99b407cbc2175c2). We report the AP on moderate condition following the KITTI benchmark
and compare average AP over all classes on moderate condition for performance on 3 classes.
and compare average AP over all classes on moderate condition for performance on 3 classes.
Note that we follow the original folder names for clear organization. Please rename the raw folders as shown above.
To prepare scannet data, please see [scannet](../data/scannet/README.md).
Note that we follow the original folder names for clear organization. Please rename the raw folders as shown above.
To prepare sunrgbd data, please see [sunrgbd](../data/sunrgbd/README.md).
To prepare scannet data, please see [scannet](https://github.com/open-mmlab/mmdetection3d/data/scannet/README.md).
To prepare sunrgbd data, please see [sunrgbd](https://github.com/open-mmlab/mmdetection3d/data/sunrgbd/README.md).
For using custom datasets, please refer to [Tutorials 2: Adding New Dataset](tutorials/new_dataset.md).
For using custom datasets, please refer to [Tutorials 2: Adding New Dataset](tutorials/new_dataset.md).
...
@@ -112,71 +115,70 @@ Optional arguments:
...
@@ -112,71 +115,70 @@ Optional arguments:
-`--show`: If specified, detection results will be plotted in the silient mode. It is only applicable to single GPU testing and used for debugging and visualization. This should be used with `--show-dir`.
-`--show`: If specified, detection results will be plotted in the silient mode. It is only applicable to single GPU testing and used for debugging and visualization. This should be used with `--show-dir`.
-`--show-dir`: If specified, detection results will be plotted on the `***_points.obj` and `***_pred.ply` files in the specified directory. It is only applicable to single GPU testing and used for debugging and visualization. You do NOT need a GUI available in your environment for using this option.
-`--show-dir`: If specified, detection results will be plotted on the `***_points.obj` and `***_pred.ply` files in the specified directory. It is only applicable to single GPU testing and used for debugging and visualization. You do NOT need a GUI available in your environment for using this option.
Examples:
Examples:
Assume that you have already downloaded the checkpoints to the directory `checkpoints/`.
Assume that you have already downloaded the checkpoints to the directory `checkpoints/`.
1. Test votenet on ScanNet and save the points and prediction visualization results.
1. Test votenet on ScanNet and save the points and prediction visualization results.
A notebook demo can be found in [demo/inference_demo.ipynb](https://github.com/open-mmlab/mmdetection/blob/master/demo/inference_demo.ipynb).
A notebook demo can be found in [demo/inference_demo.ipynb](https://github.com/open-mmlab/mmdetection/blob/master/demo/inference_demo.ipynb).
#### Asynchronous interface - supported for Python 3.7+
Async interface allows not to block CPU on GPU bound inference code and enables better CPU/GPU utilization for single threaded application. Inference can be done concurrently either between different input data samples or between different models of some inference pipeline.
See `tests/async_benchmark.py` to compare the speed of synchronous and asynchronous interfaces.
```python
import asyncio
import torch
from mmdet.apis import init_detector, async_inference_detector
from mmdet.utils.contextmanagers import concurrent
MMDetection implements distributed training and non-distributed training,
MMDetection implements distributed training and non-distributed training,
...
@@ -294,7 +253,7 @@ By default we evaluate the model on the validation set after each epoch, you can
...
@@ -294,7 +253,7 @@ By default we evaluate the model on the validation set after each epoch, you can
evaluation = dict(interval=12) # This evaluate the model per 12 epoch.
evaluation = dict(interval=12) # This evaluate the model per 12 epoch.
```
```
**\*Important\***: The default learning rate in config files is for 8 GPUs and 2 img/gpu (batch size = 8*2 = 16).
**\*Important\***: The default learning rate in config files is for 8 GPUs and the exact batch size is marked by the config's file name, e.g. '2x8' means 2 samples per GPU using 8 GPUs.
According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you need to set the learning rate proportional to the batch size if you use different GPUs or images per GPU, e.g., lr=0.01 for 4 GPUs * 2 img/gpu and lr=0.08 for 16 GPUs * 4 img/gpu.
According to the [Linear Scaling Rule](https://arxiv.org/abs/1706.02677), you need to set the learning rate proportional to the batch size if you use different GPUs or images per GPU, e.g., lr=0.01 for 4 GPUs * 2 img/gpu and lr=0.08 for 16 GPUs * 4 img/gpu.
### Train with a single GPU
### Train with a single GPU
...
@@ -316,6 +275,7 @@ Optional arguments are:
...
@@ -316,6 +275,7 @@ Optional arguments are:
- `--no-validate` (**not suggested**): By default, the codebase will perform evaluation at every k (default value is 1, which can be modified like [this](https://github.com/open-mmlab/mmdetection/blob/master/configs/mask_rcnn/mask_rcnn_r50_fpn_1x_coco.py#L174)) epochs during the training. To disable this behavior, use `--no-validate`.
- `--no-validate` (**not suggested**): By default, the codebase will perform evaluation at every k (default value is 1, which can be modified like [this](https://github.com/open-mmlab/mmdetection/blob/master/configs/mask_rcnn/mask_rcnn_r50_fpn_1x_coco.py#L174)) epochs during the training. To disable this behavior, use `--no-validate`.
- `--work-dir ${WORK_DIR}`: Override the working directory specified in the config file.
- `--work-dir ${WORK_DIR}`: Override the working directory specified in the config file.
- `--resume-from ${CHECKPOINT_FILE}`: Resume from a previous checkpoint file.
- `--resume-from ${CHECKPOINT_FILE}`: Resume from a previous checkpoint file.
- `--options 'Key=value'`: Overide some settings in the used config.
Difference between `resume-from` and `load-from`:
Difference between `resume-from` and `load-from`:
`resume-from` loads both the model weights and optimizer status, and the epoch is also inherited from the specified checkpoint. It is usually used for resuming the training process that is interrupted accidentally.
`resume-from` loads both the model weights and optimizer status, and the epoch is also inherited from the specified checkpoint. It is usually used for resuming the training process that is interrupted accidentally.
If you use launch training jobs with Slurm, you need to modify the config files (usually the 6th line from the bottom in config files) to set different communication ports.
If you use launch training jobs with Slurm, there are two ways to specify the ports.
In `config1.py`,
1. Modify the config files (usually the 6th line from the bottom in config files) to set different communication ports.
```python
dist_params = dict(backend='nccl', port=29500)
```
In `config2.py`,
In `config1.py`,
```python
dist_params = dict(backend='nccl', port=29501)
```
Then you can launch two jobs with `config1.py` ang `config2.py`.
The final output filename will be `faster_rcnn_r50_fpn_1x_20190801-{hash id}.pth`.
The final output filename will be `faster_rcnn_r50_fpn_1x_20190801-{hash id}.pth`.
### Test the robustness of detectors
Please refer to [robustness_benchmarking.md](robustness_benchmarking.md).
### Convert to ONNX (experimental)
We provide a script to convert model to [ONNX](https://github.com/onnx/onnx) format. The converted model could be visualized by tools like [Netron](https://github.com/lutzroeder/netron).
**Note**: This tool is still experimental. Customized operators are not supported for now. We set `use_torchvision=True` on-the-fly for `RoIPool` and `RoIAlign`.
## Tutorials
## Tutorials
Currently, we provide four tutorials for users to [finetune models](tutorials/finetune.md), [add new dataset](tutorials/new_dataset.md), [design data pipeline](tutorials/data_pipeline.md) and [add new modules](tutorials/new_modules.md).
Currently, we provide four tutorials for users to [finetune models](tutorials/finetune.md), [add new dataset](tutorials/new_dataset.md), [design data pipeline](tutorials/data_pipeline.md) and [add new modules](tutorials/new_modules.md).
Detectors pre-trained on the COCO dataset can serve as a good pre-trained model for other datasets, e.g., CityScapes and KITTI Dataset.
Detectors pre-trained on the COCO dataset can serve as a good pre-trained model for other datasets, e.g., CityScapes and KITTI Dataset.
This tutorial provides instruction for users to use the models provided in the [Model Zoo](../model_zoo.md) for other datasets to obatin better performance.
This tutorial provides instruction for users to use the models provided in the [Model Zoo](../model_zoo.md) for other datasets to obtain better performance.
There are two steps to finetune a model on a new dataset.
There are two steps to finetune a model on a new dataset.
- Add support for the new dataset following [Tutorial 2: Adding New Dataset](new_dataset.md).
- Add support for the new dataset following [Tutorial 2: Adding New Dataset](new_dataset.md).
...
@@ -11,7 +11,7 @@ There are two steps to finetune a model on a new dataset.
...
@@ -11,7 +11,7 @@ There are two steps to finetune a model on a new dataset.
Take the finetuning process on Cityscapes Dataset as an example, the users need to modify five parts in the config.
Take the finetuning process on Cityscapes Dataset as an example, the users need to modify five parts in the config.
## Inherit base configs
## Inherit base configs
To release the burdun and reduce bugs in writing the whole configs, MMDetection V2.0 support inheriting configs from multiple existing configs. To finetune a Mask RCNN model, the new config needs to inherit
To release the burden and reduce bugs in writing the whole configs, MMDetection V2.0 support inheriting configs from multiple existing configs. To finetune a Mask RCNN model, the new config needs to inherit
`_base_/models/mask_rcnn_r50_fpn.py` to build the basic structure of the model. To use the Cityscapes Dataset, the new config can also simply inherit `_base_/datasets/cityscapes_instance.py`. For runtime settings such as training schedules, the new config needs to inherit `_base_/default_runtime.py`. This configs are in the `configs` directory and the users can also choose to write the whole contents rather than use inheritance.
`_base_/models/mask_rcnn_r50_fpn.py` to build the basic structure of the model. To use the Cityscapes Dataset, the new config can also simply inherit `_base_/datasets/cityscapes_instance.py`. For runtime settings such as training schedules, the new config needs to inherit `_base_/default_runtime.py`. This configs are in the `configs` directory and the users can also choose to write the whole contents rather than use inheritance.
@@ -215,7 +215,7 @@ Currently it supports to concat and repeat datasets.
...
@@ -215,7 +215,7 @@ Currently it supports to concat and repeat datasets.
### Repeat dataset
### Repeat dataset
We use `RepeatDataset` as warpper to repeat the dataset. For example, suppose the original dataset is `Dataset_A`, to repeat it, the config looks like the following
We use `RepeatDataset` as wrapper to repeat the dataset. For example, suppose the original dataset is `Dataset_A`, to repeat it, the config looks like the following
```python
```python
dataset_A_train=dict(
dataset_A_train=dict(
type='RepeatDataset',
type='RepeatDataset',
...
@@ -228,7 +228,26 @@ dataset_A_train = dict(
...
@@ -228,7 +228,26 @@ dataset_A_train = dict(
)
)
```
```
### Concatemate dataset
### Class balanced dataset
We use `ClassBalancedDataset` as wrapper to repeat the dataset based on category
frequency. The dataset to repeat needs to instantiate function `self.get_cat_ids(idx)`
to support `ClassBalancedDataset`.
For example, to repeat `Dataset_A` with `oversample_thr=1e-3`, the config looks like the following
```python
dataset_A_train=dict(
type='ClassBalancedDataset',
oversample_thr=1e-3,
dataset=dict(# This is the original config of Dataset_A
type='Dataset_A',
...
pipeline=train_pipeline
)
)
```
You may refer to [source code](../../mmdet/datasets/dataset_wrappers.py) for details.
