get_started.md 11.4 KB
Newer Older
Xiang Xu's avatar
Xiang Xu committed
1
# Get Started
2

Xiang Xu's avatar
Xiang Xu committed
3
## Prerequisites
zhangwenwei's avatar
zhangwenwei committed
4

Xiang Xu's avatar
Xiang Xu committed
5
6
7
In this section, we demonstrate how to prepare an environment with PyTorch.

MMDetection3D works on Linux, Windows (experimental support) and macOS. It requires Python 3.7+, CUDA 9.2+, and PyTorch 1.6+.
xiliu8006's avatar
xiliu8006 committed
8

9
10
```{note}
If you are experienced with PyTorch and have already installed it, just skip this part and jump to the [next section](#installation). Otherwise, you can follow these steps for the preparation.
Wenwei Zhang's avatar
Wenwei Zhang committed
11
12
```

13
**Step 0.** Download and install Miniconda from the [official website](https://docs.conda.io/en/latest/miniconda.html).
Wenwei Zhang's avatar
Wenwei Zhang committed
14

15
**Step 1.** Create a conda environment and activate it.
Wenwei Zhang's avatar
Wenwei Zhang committed
16

17
```shell
18
19
conda create --name openmmlab python=3.8 -y
conda activate openmmlab
Wenwei Zhang's avatar
Wenwei Zhang committed
20
21
```

22
**Step 2.** Install PyTorch following [official instructions](https://pytorch.org/get-started/locally/), e.g.
zhangwenwei's avatar
zhangwenwei committed
23

24
On GPU platforms:
zhangwenwei's avatar
zhangwenwei committed
25

Ziyi Wu's avatar
Ziyi Wu committed
26
```shell
27
conda install pytorch torchvision -c pytorch
xiliu8006's avatar
xiliu8006 committed
28
29
```

30
On CPU platforms:
xiliu8006's avatar
xiliu8006 committed
31

twang's avatar
twang committed
32
```shell
33
conda install pytorch torchvision cpuonly -c pytorch
twang's avatar
twang committed
34
```
zhangwenwei's avatar
zhangwenwei committed
35

Xiang Xu's avatar
Xiang Xu committed
36
## Installation
37

38
We recommend that users follow our best practices to install MMDetection3D. However, the whole process is highly customizable. See [Customize Installation](#customize-installation) section for more information.
39

Xiang Xu's avatar
Xiang Xu committed
40
### Best Practices
41

Xiang Xu's avatar
Xiang Xu committed
42
**Step 0.** Install [MMEngine](https://github.com/open-mmlab/mmengine), [MMCV](https://github.com/open-mmlab/mmcv) and [MMDetection](https://github.com/open-mmlab/mmdetection) using [MIM](https://github.com/open-mmlab/mim).
zhangwenwei's avatar
zhangwenwei committed
43

twang's avatar
twang committed
44
```shell
45
pip install -U openmim
46
mim install mmengine
Xiang Xu's avatar
Xiang Xu committed
47
mim install 'mmcv>=2.0.0rc1'
48
mim install 'mmdet>=3.0.0rc0'
xiliu8006's avatar
xiliu8006 committed
49
50
```

Xiang Xu's avatar
Xiang Xu committed
51
**Note**: In MMCV-v2.x, `mmcv-full` is renamed to `mmcv`, if you want to install `mmcv` without CUDA ops, you can use `mim install "mmcv-lite>=2.0.0rc1"` to install the lite version.
zhangwenwei's avatar
zhangwenwei committed
52

Xiang Xu's avatar
Xiang Xu committed
53
**Step 1.** Install MMDetection3D.
zhangwenwei's avatar
zhangwenwei committed
54

Xiang Xu's avatar
Xiang Xu committed
55
Case a: If you develop and run mmdet3d directly, install it from source:
zhangwenwei's avatar
Doc  
zhangwenwei committed
56

twang's avatar
twang committed
57
```shell
58
59
git clone https://github.com/open-mmlab/mmdetection3d.git -b dev-1.x
# "-b dev-1.x" means checkout to the `dev-1.x` branch.
twang's avatar
twang committed
60
cd mmdetection3d
Xiang Xu's avatar
Xiang Xu committed
61
62
63
64
pip install -v -e .
# "-v" means verbose, or more output
# "-e" means installing a project in edtiable mode,
# thus any local modifications made to the code will take effect without reinstallation.
twang's avatar
twang committed
65
```
zhangwenwei's avatar
zhangwenwei committed
66

Xiang Xu's avatar
Xiang Xu committed
67
Case b: If you use mmdet3d as a dependency or third-party package, install it with MIM:
zhangwenwei's avatar
zhangwenwei committed
68

twang's avatar
twang committed
69
```shell
Xiang Xu's avatar
Xiang Xu committed
70
mim install "mmdet3d>=1.1.0rc0"
twang's avatar
twang committed
71
```
zhangwenwei's avatar
zhangwenwei committed
72

twang's avatar
twang committed
73
Note:
zhangwenwei's avatar
Doc  
zhangwenwei committed
74

Xiang Xu's avatar
Xiang Xu committed
75
1. If you would like to use `opencv-python-headless` instead of `opencv-python`,
76
   you can install it before installing MMCV.
zhangwenwei's avatar
zhangwenwei committed
77

Xiang Xu's avatar
Xiang Xu committed
78
2. Some dependencies are optional. Simply running `pip install -v -e .` will only install the minimum runtime requirements. To use optional dependencies like `albumentations` and `imagecorruptions` either install them manually with `pip install -r requirements/optional.txt` or specify desired extras when calling `pip` (e.g. `pip install -v -e .[optional]`). Valid keys for the extras field are: `all`, `tests`, `build`, and `optional`.
zhangwenwei's avatar
zhangwenwei committed
79

80
   We have supported `spconv 2.0`. If the user has installed `spconv 2.0`, the code will use `spconv 2.0` first, which will take up less GPU memory than using the default `mmcv spconv`. Users can use the following commands to install `spconv 2.0`:
VVsssssk's avatar
VVsssssk committed
81

Xiang Xu's avatar
Xiang Xu committed
82
   ```shell
VVsssssk's avatar
VVsssssk committed
83
84
85
86
   pip install cumm-cuxxx
   pip install spconv-cuxxx
   ```

87
   Where `xxx` is the CUDA version in the environment.
VVsssssk's avatar
VVsssssk committed
88
89
90
91
92

   For example, using CUDA 10.2, the command will be `pip install cumm-cu102 && pip install spconv-cu102`.

   Supported CUDA versions include 10.2, 11.1, 11.3, and 11.4. Users can also install it by building from the source. For more details please refer to [spconv v2.x](https://github.com/traveller59/spconv).

93
   We also support `Minkowski Engine` as a sparse convolution backend. If necessary please follow original [installation guide](https://github.com/NVIDIA/MinkowskiEngine#installation) or use `pip` to install it:
94

95
96
97
98
   ```shell
   conda install openblas-devel -c anaconda
   pip install -U git+https://github.com/NVIDIA/MinkowskiEngine -v --no-deps --install-option="--blas_include_dirs=/opt/conda/include" --install-option="--blas=openblas"
   ```
99

Xiang Xu's avatar
Xiang Xu committed
100
3. The code can not be built for CPU only environment (where CUDA isn't available) for now.
zhangwenwei's avatar
zhangwenwei committed
101

Xiang Xu's avatar
Xiang Xu committed
102
### Verify the Installation
Wenwei Zhang's avatar
Wenwei Zhang committed
103

Xiang Xu's avatar
Xiang Xu committed
104
To verify whether MMDetection3D is installed correctly, we provide some sample codes to run an inference demo.
zhangwenwei's avatar
Doc  
zhangwenwei committed
105

Xiang Xu's avatar
Xiang Xu committed
106
**Step 1.** We need to download config and checkpoint files.
zhangwenwei's avatar
Doc  
zhangwenwei committed
107
108

```shell
Xiang Xu's avatar
Xiang Xu committed
109
mim download mmdet3d --config pointpillars_hv_secfpn_8xb6-160e_kitti-3d-car --dest .
zhangwenwei's avatar
Doc  
zhangwenwei committed
110
111
```

Xiang Xu's avatar
Xiang Xu committed
112
113
114
115
116
The downloading will take several seconds or more, depending on your network environment. When it is done, you will find two files `pointpillars_hv_secfpn_8xb6-160e_kitti-3d-car.py` and `hv_pointpillars_secfpn_6x8_160e_kitti-3d-car_20220331_134606-d42d15ed.pth` in your current folder.

**Step 2.** Verify the inference demo.

Case a: If you install MMDetection3D from source, just run the following command.
zhangwenwei's avatar
Doc  
zhangwenwei committed
117
118

```shell
Xiang Xu's avatar
Xiang Xu committed
119
python demo/pcd_demo.py demo/data/kitti/000008.bin pointpillars_hv_secfpn_8xb6-160e_kitti-3d-car.py hv_pointpillars_secfpn_6x8_160e_kitti-3d-car_20220331_134606-d42d15ed.pth --show
zhangwenwei's avatar
zhangwenwei committed
120
```
121

Xiang Xu's avatar
Xiang Xu committed
122
123
124
125
You will see a visualizer interface with point cloud, where bounding boxes are plotted on cars.

**Note**:

126
If you want to input a `.ply` file, you can use the following function and convert it to `.bin` format. Then you can use the converted `.bin` file to run demo.
127
Note that you need to install `pandas` and `plyfile` before using this script. This function can also be used for data preprocessing for training `ply data`.
128

yinchimaoliang's avatar
yinchimaoliang committed
129
130
131
132
133
```python
import numpy as np
import pandas as pd
from plyfile import PlyData

134
def convert_ply(input_path, output_path):
yinchimaoliang's avatar
yinchimaoliang committed
135
136
137
138
139
140
141
142
143
144
    plydata = PlyData.read(input_path)  # read file
    data = plydata.elements[0].data  # read data
    data_pd = pd.DataFrame(data)  # convert to DataFrame
    data_np = np.zeros(data_pd.shape, dtype=np.float)  # initialize array to store data
    property_names = data[0].dtype.names  # read names of properties
    for i, name in enumerate(
            property_names):  # read data by property
        data_np[:, i] = data_pd[name]
    data_np.astype(np.float32).tofile(output_path)
```
145

yinchimaoliang's avatar
yinchimaoliang committed
146
Examples:
zhangwenwei's avatar
zhangwenwei committed
147

yinchimaoliang's avatar
yinchimaoliang committed
148
149
150
```python
convert_ply('./test.ply', './test.bin')
```
zhangwenwei's avatar
zhangwenwei committed
151

152
If you have point clouds in other format (`.off`, `.obj`, etc.), you can use `trimesh` to convert them into `.ply`.
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167

```python
import trimesh

def to_ply(input_path, output_path, original_type):
    mesh = trimesh.load(input_path, file_type=original_type)  # read file
    mesh.export(output_path, file_type='ply')  # convert to ply
```

Examples:

```python
to_ply('./test.obj', './test.ply', 'obj')
```

Xiang Xu's avatar
Xiang Xu committed
168
169
170
171
172
173
174
175
176
177
178
179
180
181
Case b: If you install MMDetection3D with MIM, open your python interpreter and copy&paste the following codes.

```python
from mmdet3d.apis import init_model, inference_detector
from mmdet3d.utils import register_all_modules

register_all_modules()
config_file = 'pointpillars_hv_secfpn_8xb6-160e_kitti-3d-car.py'
checkpoint_file = 'hv_pointpillars_secfpn_6x8_160e_kitti-3d-car_20220331_134606-d42d15ed.pth'
model = init_model(config_file, checkpoint_file)
inference_detector(model, 'demo/data/kitti/000008.bin')
```

You will see a list of `Det3DDataSample`, and the predictions are in the `pred_instances_3d`, indicating the detected bounding boxes, labels, and scores.
182

Xiang Xu's avatar
Xiang Xu committed
183
### Customize Installation
zhangwenwei's avatar
zhangwenwei committed
184

Xiang Xu's avatar
Xiang Xu committed
185
#### CUDA Versions
186

187
When installing PyTorch, you need to specify the version of CUDA. If you are not clear on which to choose, follow our recommendations:
Ziyi Wu's avatar
Ziyi Wu committed
188

189
190
- For Ampere-based NVIDIA GPUs, such as GeForce 30 series and NVIDIA A100, CUDA 11 is a must.
- For older NVIDIA GPUs, CUDA 11 is backward compatible, but CUDA 10.2 offers better compatibility and is more lightweight.
zhangwenwei's avatar
zhangwenwei committed
191

192
193
194
195
196
197
Please make sure the GPU driver satisfies the minimum version requirements. See [this table](https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#cuda-major-component-versions__table-cuda-toolkit-driver-versions) for more information.

```{note}
Installing CUDA runtime libraries is enough if you follow our best practices, because no CUDA code will be compiled locally. However if you hope to compile MMCV from source or develop other CUDA operators, you need to install the complete CUDA toolkit from NVIDIA's [website](https://developer.nvidia.com/cuda-downloads), and its version should match the CUDA version of PyTorch. i.e., the specified version of cudatoolkit in `conda install` command.
```

Xiang Xu's avatar
Xiang Xu committed
198
#### Install MMEngine without MIM
199

200
To install MMEngine with pip instead of MIM, please follow [MMEngine installation guides](https://mmengine.readthedocs.io/en/latest/get_started/installation.html).
201

202
For example, you can install MMEngine by the following command:
203
204
205
206
207

```shell
pip install mmengine
```

Xiang Xu's avatar
Xiang Xu committed
208
#### Install MMCV without MIM
209
210
211

MMCV contains C++ and CUDA extensions, thus depending on PyTorch in a complex way. MIM solves such dependencies automatically and makes the installation easier. However, it is not a must.

212
To install MMCV with pip instead of MIM, please follow [MMCV installation guides](https://mmcv.readthedocs.io/en/2.x/get_started/installation.html). This requires manually specifying a find-url based on PyTorch version and its CUDA version.
213

Xiang Xu's avatar
Xiang Xu committed
214
For example, the following command install MMCV built for PyTorch 1.12.x and CUDA 11.6:
215
216

```shell
Xiang Xu's avatar
Xiang Xu committed
217
pip install "mmcv>=2.0.0rc1" -f https://download.openmmlab.com/mmcv/dist/cu116/torch1.12.0/index.html
218
219
```

Xiang Xu's avatar
Xiang Xu committed
220
#### Install on Google Colab
221

Xiang Xu's avatar
Xiang Xu committed
222
223
224
[Google Colab](https://colab.research.google.com/) usually has PyTorch installed, thus we only need to install MMEngine, MMCV, MMDetection, and MMDetection3D with the following commands.

**Step 1.** Install [MMEngine](https://github.com/open-mmlab/mmengine), [MMCV](https://github.com/open-mmlab/mmcv) and [MMDetection](https://github.com/open-mmlab/mmdetection) using [MIM](https://github.com/open-mmlab/mim).
zhangwenwei's avatar
zhangwenwei committed
225

226
```shell
Xiang Xu's avatar
Xiang Xu committed
227
228
229
230
!pip3 install openmim
!mim install mmengine
!mim install "mmcv>=2.0.0rc1,<2.1.0"
!mim install "mmdet>=3.0.0rc0,<3.1.0"
231
```
zhangwenwei's avatar
zhangwenwei committed
232

Xiang Xu's avatar
Xiang Xu committed
233
**Step 2.** Install MMDetection3D from source.
zhangwenwei's avatar
zhangwenwei committed
234

235
```shell
Xiang Xu's avatar
Xiang Xu committed
236
237
238
!git clone https://github.com/open-mmlab/mmdetection3d.git -b dev-1.x
%cd mmdetection3d
!pip install -e .
zhangwenwei's avatar
zhangwenwei committed
239
```
240

Xiang Xu's avatar
Xiang Xu committed
241
**Step 3.** Verification.
242

Xiang Xu's avatar
Xiang Xu committed
243
244
245
246
247
```python
import mmdet3d
print(mmdet3d.__version__)
# Example output: 1.1.0rc0, or an another version.
```
248

Xiang Xu's avatar
Xiang Xu committed
249
250
251
```{note}
Within Jupyter, the exclamation mark `!` is used to call external executables and `%cd` is a [magic command](https://ipython.readthedocs.io/en/stable/interactive/magics.html#magic-cd) to change the current working directory of Python.
```
252

Xiang Xu's avatar
Xiang Xu committed
253
#### Using MMDetection3D with Docker
254

Xiang Xu's avatar
Xiang Xu committed
255
We provide a [Dockerfile](https://github.com/open-mmlab/mmdetection3d/blob/dev-1.x/docker/Dockerfile) to build an image. Ensure that your [docker version](https://docs.docker.com/engine/install/) >= 19.03.
256

Xiang Xu's avatar
Xiang Xu committed
257
258
259
260
261
```shell
# build an image with PyTorch 1.6, CUDA 10.1
# If you prefer other versions, just modified the Dockerfile
docker build -t mmdetection3d docker/
```
262

Xiang Xu's avatar
Xiang Xu committed
263
264
265
266
Run it with:

```shell
docker run --gpus all --shm-size=8g -it -v {DATA_DIR}:/mmdetection3d/data mmdetection3d
267
268
```

Xiang Xu's avatar
Xiang Xu committed
269
### Troubleshooting
270

271
If you have some issues during the installation, please first view the [FAQ](notes/faq.md) page.
272
You may [open an issue](https://github.com/open-mmlab/mmdetection3d/issues/new/choose) on GitHub if no solution is found.
Xiang Xu's avatar
Xiang Xu committed
273
274
275
276
277
278
279
280
281
282

### Use Multiple Versions of MMDetection3D in Development

Training and testing scripts have already been modified in `PYTHONPATH` in order to make sure the scripts are using their own versions of MMDetection3D.

To install the default version of MMDetection3D in your environment, you can exclude the following code in the related scripts:

```shell
PYTHONPATH="$(dirname $0)/..":$PYTHONPATH
```