getting_started.md 13.9 KB
Newer Older
twang's avatar
twang committed
1
# Prerequisites
zhangwenwei's avatar
zhangwenwei committed
2

3
- Linux or macOS (Windows is in experimental support)
twang's avatar
twang committed
4
5
6
7
- Python 3.6+
- PyTorch 1.3+
- CUDA 9.2+ (If you build PyTorch from source, CUDA 9.0 is also compatible)
- GCC 5+
xiliu8006's avatar
xiliu8006 committed
8
9
10
- [MMCV](https://mmcv.readthedocs.io/en/latest/#installation)


11
12
The required versions of MMCV, MMDetection and MMSegmentation for different versions of MMDetection3D are as below. Please install the correct version of MMCV, MMDetection and MMSegmentation to avoid installation issues.

VVsssssk's avatar
VVsssssk committed
13
14
| MMDetection3D version |   MMDetection version   | MMSegmentation version |        MMCV version        |
| :-------------------: | :---------------------: | :--------------------: | :------------------------: |
Tai-Wang's avatar
Tai-Wang committed
15
16
|        master         | mmdet>=2.19.0, <=3.0.0  | mmseg>=0.20.0, <=1.0.0 | mmcv-full>=1.4.8, <=1.7.0  |
|       v1.0.0rc2       | mmdet>=2.19.0, <=3.0.0  | mmseg>=0.20.0, <=1.0.0 | mmcv-full>=1.4.8, <=1.7.0  |
VVsssssk's avatar
VVsssssk committed
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
|       v1.0.0rc1       | mmdet>=2.19.0, <=3.0.0  | mmseg>=0.20.0, <=1.0.0 | mmcv-full>=1.4.8, <=1.5.0  |
|       v1.0.0rc0       | mmdet>=2.19.0, <=3.0.0  | mmseg>=0.20.0, <=1.0.0 | mmcv-full>=1.3.17, <=1.5.0 |
|        0.18.1         | mmdet>=2.19.0, <=3.0.0  | mmseg>=0.20.0, <=1.0.0 | mmcv-full>=1.3.17, <=1.5.0 |
|        0.18.0         | mmdet>=2.19.0, <=3.0.0  | mmseg>=0.20.0, <=1.0.0 | mmcv-full>=1.3.17, <=1.5.0 |
|        0.17.3         | mmdet>=2.14.0, <=3.0.0  | mmseg>=0.14.1, <=1.0.0 | mmcv-full>=1.3.8, <=1.4.0  |
|        0.17.2         | mmdet>=2.14.0, <=3.0.0  | mmseg>=0.14.1, <=1.0.0 | mmcv-full>=1.3.8, <=1.4.0  |
|        0.17.1         | mmdet>=2.14.0, <=3.0.0  | mmseg>=0.14.1, <=1.0.0 | mmcv-full>=1.3.8, <=1.4.0  |
|        0.17.0         | mmdet>=2.14.0, <=3.0.0  | mmseg>=0.14.1, <=1.0.0 | mmcv-full>=1.3.8, <=1.4.0  |
|        0.16.0         | mmdet>=2.14.0, <=3.0.0  | mmseg>=0.14.1, <=1.0.0 | mmcv-full>=1.3.8, <=1.4.0  |
|        0.15.0         | mmdet>=2.14.0, <=3.0.0  | mmseg>=0.14.1, <=1.0.0 | mmcv-full>=1.3.8, <=1.4.0  |
|        0.14.0         | mmdet>=2.10.0, <=2.11.0 |     mmseg==0.14.0      | mmcv-full>=1.3.1, <=1.4.0  |
|        0.13.0         | mmdet>=2.10.0, <=2.11.0 |      Not required      | mmcv-full>=1.2.4, <=1.4.0  |
|        0.12.0         | mmdet>=2.5.0, <=2.11.0  |      Not required      | mmcv-full>=1.2.4, <=1.4.0  |
|        0.11.0         | mmdet>=2.5.0, <=2.11.0  |      Not required      | mmcv-full>=1.2.4, <=1.3.0  |
|        0.10.0         | mmdet>=2.5.0, <=2.11.0  |      Not required      | mmcv-full>=1.2.4, <=1.3.0  |
|         0.9.0         | mmdet>=2.5.0, <=2.11.0  |      Not required      | mmcv-full>=1.2.4, <=1.3.0  |
|         0.8.0         | mmdet>=2.5.0, <=2.11.0  |      Not required      | mmcv-full>=1.1.5, <=1.3.0  |
|         0.7.0         | mmdet>=2.5.0, <=2.11.0  |      Not required      | mmcv-full>=1.1.5, <=1.3.0  |
|         0.6.0         | mmdet>=2.4.0, <=2.11.0  |      Not required      | mmcv-full>=1.1.3, <=1.2.0  |
|         0.5.0         |          2.3.0          |      Not required      |      mmcv-full==1.0.5      |
zhangwenwei's avatar
Doc  
zhangwenwei committed
37

twang's avatar
twang committed
38
# Installation
zhangwenwei's avatar
Doc  
zhangwenwei committed
39

twang's avatar
twang committed
40
## Install MMDetection3D
zhangwenwei's avatar
Doc  
zhangwenwei committed
41

42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
### Quick installation instructions script

Assuming that you already have CUDA 11.0 installed, here is a full script for quick installation of MMDetection3D with conda.
Otherwise, you should refer to the step-by-step installation instructions in the next section.

```shell
conda create -n open-mmlab python=3.7 pytorch=1.9 cudatoolkit=11.0 torchvision -c pytorch -y
conda activate open-mmlab
pip3 install openmim
mim install mmcv-full
mim install mmdet
mim install mmsegmentation
git clone https://github.com/open-mmlab/mmdetection3d.git
cd mmdetection3d
pip3 install -e .
```

### Step-by-step installation instructions

61
**a. Create a conda virtual environment and activate it.**
zhangwenwei's avatar
zhangwenwei committed
62

twang's avatar
twang committed
63
64
65
```shell
conda create -n open-mmlab python=3.7 -y
conda activate open-mmlab
zhangwenwei's avatar
Doc  
zhangwenwei committed
66
67
```

68
**b. Install PyTorch and torchvision following the [official instructions](https://pytorch.org/).**
Wenwei Zhang's avatar
Wenwei Zhang committed
69

twang's avatar
twang committed
70
71
```shell
conda install pytorch torchvision -c pytorch
Wenwei Zhang's avatar
Wenwei Zhang committed
72
73
```

twang's avatar
twang committed
74
75
Note: Make sure that your compilation CUDA version and runtime CUDA version match.
You can check the supported CUDA version for precompiled packages on the [PyTorch website](https://pytorch.org/).
Wenwei Zhang's avatar
Wenwei Zhang committed
76

77
`E.g. 1` If you have CUDA 10.1 installed under `/usr/local/cuda` and would like to install
twang's avatar
twang committed
78
PyTorch 1.5, you need to install the prebuilt PyTorch with CUDA 10.1.
Wenwei Zhang's avatar
Wenwei Zhang committed
79

80
```shell
81
conda install pytorch==1.5.0 cudatoolkit=10.1 torchvision==0.6.0 -c pytorch
Wenwei Zhang's avatar
Wenwei Zhang committed
82
83
```

twang's avatar
twang committed
84
85
`E.g. 2` If you have CUDA 9.2 installed under `/usr/local/cuda` and would like to install
PyTorch 1.3.1., you need to install the prebuilt PyTorch with CUDA 9.2.
zhangwenwei's avatar
zhangwenwei committed
86

87
```shell
twang's avatar
twang committed
88
conda install pytorch=1.3.1 cudatoolkit=9.2 torchvision=0.4.2 -c pytorch
wangtai's avatar
wangtai committed
89
90
```

91
If you build PyTorch from source instead of installing the prebuilt package,
twang's avatar
twang committed
92
you can use more CUDA versions such as 9.0.
93

94
**c. Install [MMCV](https://mmcv.readthedocs.io/en/latest/).**
xiliu8006's avatar
xiliu8006 committed
95
*mmcv-full* is necessary since MMDetection3D relies on MMDetection, CUDA ops in *mmcv-full* are required.
zhangwenwei's avatar
Doc  
zhangwenwei committed
96

97
`e.g.` The pre-build *mmcv-full* could be installed by running: (available versions could be found [here](https://mmcv.readthedocs.io/en/latest/#install-with-pip))
zhangwenwei's avatar
zhangwenwei committed
98

Ziyi Wu's avatar
Ziyi Wu committed
99
```shell
xiliu8006's avatar
xiliu8006 committed
100
101
102
103
104
pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/{cu_version}/{torch_version}/index.html
```

Please replace `{cu_version}` and `{torch_version}` in the url to your desired one. For example, to install the latest `mmcv-full` with `CUDA 11` and `PyTorch 1.7.0`, use the following command:

twang's avatar
twang committed
105
```shell
xiliu8006's avatar
xiliu8006 committed
106
pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu110/torch1.7.0/index.html
twang's avatar
twang committed
107
```
zhangwenwei's avatar
zhangwenwei committed
108

109
110
111
112
113
114
115
mmcv-full is only compiled on PyTorch 1.x.0 because the compatibility usually holds between 1.x.0 and 1.x.1. If your PyTorch version is 1.x.1, you can install mmcv-full compiled with PyTorch 1.x.0 and it usually works well.

```shell
# We can ignore the micro version of PyTorch
pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu110/torch1.7/index.html
```

xiliu8006's avatar
xiliu8006 committed
116
See [here](https://github.com/open-mmlab/mmcv#install-with-pip) for different versions of MMCV compatible to different PyTorch and CUDA versions.
twang's avatar
twang committed
117
Optionally, you could also build the full version from source:
zhangwenwei's avatar
zhangwenwei committed
118

twang's avatar
twang committed
119
```shell
xiliu8006's avatar
xiliu8006 committed
120
121
122
123
124
125
126
127
128
129
git clone https://github.com/open-mmlab/mmcv.git
cd mmcv
MMCV_WITH_OPS=1 pip install -e .  # package mmcv-full will be installed after this step
cd ..
```

Or directly run

```shell
pip install mmcv-full
twang's avatar
twang committed
130
```
zhangwenwei's avatar
zhangwenwei committed
131

132
**d. Install [MMDetection](https://github.com/open-mmlab/mmdetection).**
zhangwenwei's avatar
zhangwenwei committed
133

twang's avatar
twang committed
134
```shell
135
pip install mmdet
twang's avatar
twang committed
136
```
zhangwenwei's avatar
zhangwenwei committed
137

twang's avatar
twang committed
138
Optionally, you could also build MMDetection from source in case you want to modify the code:
zhangwenwei's avatar
zhangwenwei committed
139
140

```shell
twang's avatar
twang committed
141
142
git clone https://github.com/open-mmlab/mmdetection.git
cd mmdetection
143
git checkout v2.19.0  # switch to v2.19.0 branch
twang's avatar
twang committed
144
145
pip install -r requirements/build.txt
pip install -v -e .  # or "python setup.py develop"
zhangwenwei's avatar
zhangwenwei committed
146
147
```

148
149
150
**e. Install [MMSegmentation](https://github.com/open-mmlab/mmsegmentation).**

```shell
151
pip install mmsegmentation
152
153
154
155
156
157
158
```

Optionally, you could also build MMSegmentation from source in case you want to modify the code:

```shell
git clone https://github.com/open-mmlab/mmsegmentation.git
cd mmsegmentation
159
git checkout v0.20.0  # switch to v0.20.0 branch
160
161
162
163
pip install -e .  # or "python setup.py develop"
```

**f. Clone the MMDetection3D repository.**
zhangwenwei's avatar
Doc  
zhangwenwei committed
164

twang's avatar
twang committed
165
166
167
168
```shell
git clone https://github.com/open-mmlab/mmdetection3d.git
cd mmdetection3d
```
zhangwenwei's avatar
zhangwenwei committed
169

170
**g.Install build requirements and then install MMDetection3D.**
zhangwenwei's avatar
zhangwenwei committed
171

twang's avatar
twang committed
172
173
174
```shell
pip install -v -e .  # or "python setup.py develop"
```
zhangwenwei's avatar
zhangwenwei committed
175

twang's avatar
twang committed
176
Note:
zhangwenwei's avatar
Doc  
zhangwenwei committed
177

twang's avatar
twang committed
178
179
1. The git commit id will be written to the version number with step d, e.g. 0.6.0+2e7045c. The version will also be saved in trained models.
It is recommended that you run step d each time you pull some updates from github. If C++/CUDA codes are modified, then this step is compulsory.
zhangwenwei's avatar
Doc  
zhangwenwei committed
180

twang's avatar
twang committed
181
    > Important: Be sure to remove the `./build` folder if you reinstall mmdet with a different CUDA/PyTorch version.
zhangwenwei's avatar
zhangwenwei committed
182

twang's avatar
twang committed
183
184
185
186
187
    ```shell
    pip uninstall mmdet3d
    rm -rf ./build
    find . -name "*.so" | xargs rm
    ```
zhangwenwei's avatar
zhangwenwei committed
188

189
2. Following the above instructions, MMDetection3D is installed on `dev` mode, any local modifications made to the code will take effect without the need to reinstall it (unless you submit some commits and want to update the version number).
zhangwenwei's avatar
zhangwenwei committed
190

twang's avatar
twang committed
191
192
3. If you would like to use `opencv-python-headless` instead of `opencv-python`,
you can install it before installing MMCV.
zhangwenwei's avatar
zhangwenwei committed
193

twang's avatar
twang committed
194
4. Some dependencies are optional. Simply running `pip install -v -e .` will only install the minimum runtime requirements. To use optional dependencies like `albumentations` and `imagecorruptions` either install them manually with `pip install -r requirements/optional.txt` or specify desired extras when calling `pip` (e.g. `pip install -v -e .[optional]`). Valid keys for the extras field are: `all`, `tests`, `build`, and `optional`.
zhangwenwei's avatar
zhangwenwei committed
195

VVsssssk's avatar
VVsssssk committed
196
197
198
199
200
201
202
203
204
205
206
207
208
   We have supported spconv2.0. If the user has installed spconv2.0, the code will use spconv2.0 first, which will take up less GPU memory than using the default mmcv spconv. Users can use the following commands to install spconv2.0:

   ```bash
   pip install cumm-cuxxx
   pip install spconv-cuxxx
   ```

   Where xxx is the CUDA version in the environment.

   For example, using CUDA 10.2, the command will be `pip install cumm-cu102 && pip install spconv-cu102`.

   Supported CUDA versions include 10.2, 11.1, 11.3, and 11.4. Users can also install it by building from the source. For more details please refer to [spconv v2.x](https://github.com/traveller59/spconv).

209
210
211
212
213
214
215
   We also support Minkowski Engine as a sparse convolution backend. If necessary please follow original [installation guide](https://github.com/NVIDIA/MinkowskiEngine#installation) or use `pip`:

    ```shell
    conda install openblas-devel -c anaconda
    pip install -U git+https://github.com/NVIDIA/MinkowskiEngine -v --no-deps --install-option="--blas_include_dirs=/opt/conda/include" --install-option="--blas=openblas"
    ```

twang's avatar
twang committed
216
5. The code can not be built for CPU only environment (where CUDA isn't available) for now.
zhangwenwei's avatar
zhangwenwei committed
217

twang's avatar
twang committed
218
## Another option: Docker Image
Wenwei Zhang's avatar
Wenwei Zhang committed
219

twang's avatar
twang committed
220
We provide a [Dockerfile](https://github.com/open-mmlab/mmdetection3d/blob/master/docker/Dockerfile) to build an image.
Wenwei Zhang's avatar
Wenwei Zhang committed
221

twang's avatar
twang committed
222
223
```shell
# build an image with PyTorch 1.6, CUDA 10.1
224
docker build -t mmdetection3d -f docker/Dockerfile .
twang's avatar
twang committed
225
```
Wenwei Zhang's avatar
Wenwei Zhang committed
226

twang's avatar
twang committed
227
Run it with
Wenwei Zhang's avatar
Wenwei Zhang committed
228

twang's avatar
twang committed
229
230
231
```shell
docker run --gpus all --shm-size=8g -it -v {DATA_DIR}:/mmdetection3d/data mmdetection3d
```
Wenwei Zhang's avatar
Wenwei Zhang committed
232

twang's avatar
twang committed
233
## A from-scratch setup script
Wenwei Zhang's avatar
Wenwei Zhang committed
234

235
Here is a full script for setting up MMdetection3D with conda.
Wenwei Zhang's avatar
Wenwei Zhang committed
236

twang's avatar
twang committed
237
238
239
```shell
conda create -n open-mmlab python=3.7 -y
conda activate open-mmlab
Wenwei Zhang's avatar
Wenwei Zhang committed
240

241
# install latest PyTorch prebuilt with the default prebuilt CUDA version (usually the latest)
twang's avatar
twang committed
242
conda install -c pytorch pytorch torchvision -y
Wenwei Zhang's avatar
Wenwei Zhang committed
243

twang's avatar
twang committed
244
245
# install mmcv
pip install mmcv-full
liyinhao's avatar
liyinhao committed
246

twang's avatar
twang committed
247
248
# install mmdetection
pip install git+https://github.com/open-mmlab/mmdetection.git
liyinhao's avatar
liyinhao committed
249

250
251
252
# install mmsegmentation
pip install git+https://github.com/open-mmlab/mmsegmentation.git

twang's avatar
twang committed
253
254
255
256
# install mmdetection3d
git clone https://github.com/open-mmlab/mmdetection3d.git
cd mmdetection3d
pip install -v -e .
zhangwenwei's avatar
zhangwenwei committed
257
```
liyinhao's avatar
liyinhao committed
258

twang's avatar
twang committed
259
260
261
## Using multiple MMDetection3D versions

The train and test scripts already modify the `PYTHONPATH` to ensure the script use the MMDetection3D in the current directory.
liyinhao's avatar
liyinhao committed
262

twang's avatar
twang committed
263
264
265
266
To use the default MMDetection3D installed in the environment rather than that you are working with, you can remove the following line in those scripts

```shell
PYTHONPATH="$(dirname $0)/..":$PYTHONPATH
liyinhao's avatar
liyinhao committed
267
268
```

twang's avatar
twang committed
269
# Verification
liyinhao's avatar
liyinhao committed
270

271
## Verify with point cloud demo
zhangwenwei's avatar
Doc  
zhangwenwei committed
272

273
We provide several demo scripts to test a single sample. Pre-trained models can be downloaded from [model zoo](model_zoo.md). To test a single-modality 3D detection on point cloud scenes:
zhangwenwei's avatar
Doc  
zhangwenwei committed
274
275

```shell
wuyuefeng's avatar
Demo  
wuyuefeng committed
276
python demo/pcd_demo.py ${PCD_FILE} ${CONFIG_FILE} ${CHECKPOINT_FILE} [--device ${GPU_ID}] [--score-thr ${SCORE_THR}] [--out-dir ${OUT_DIR}]
zhangwenwei's avatar
Doc  
zhangwenwei committed
277
278
279
280
281
```

Examples:

```shell
282
python demo/pcd_demo.py demo/data/kitti/kitti_000008.bin configs/second/hv_second_secfpn_6x8_80e_kitti-3d-car.py checkpoints/hv_second_secfpn_6x8_80e_kitti-3d-car_20200620_230238-393f000c.pth
zhangwenwei's avatar
zhangwenwei committed
283
```
284

yinchimaoliang's avatar
yinchimaoliang committed
285
If you want to input a `ply` file, you can use the following function and convert it to `bin` format. Then you can use the converted `bin` file to generate demo.
286
Note that you need to install `pandas` and `plyfile` before using this script. This function can also be used for data preprocessing for training ```ply data```.
287

yinchimaoliang's avatar
yinchimaoliang committed
288
289
290
291
292
```python
import numpy as np
import pandas as pd
from plyfile import PlyData

293
def convert_ply(input_path, output_path):
yinchimaoliang's avatar
yinchimaoliang committed
294
295
296
297
298
299
300
301
302
303
    plydata = PlyData.read(input_path)  # read file
    data = plydata.elements[0].data  # read data
    data_pd = pd.DataFrame(data)  # convert to DataFrame
    data_np = np.zeros(data_pd.shape, dtype=np.float)  # initialize array to store data
    property_names = data[0].dtype.names  # read names of properties
    for i, name in enumerate(
            property_names):  # read data by property
        data_np[:, i] = data_pd[name]
    data_np.astype(np.float32).tofile(output_path)
```
304

yinchimaoliang's avatar
yinchimaoliang committed
305
Examples:
zhangwenwei's avatar
zhangwenwei committed
306

yinchimaoliang's avatar
yinchimaoliang committed
307
308
309
```python
convert_ply('./test.ply', './test.bin')
```
zhangwenwei's avatar
zhangwenwei committed
310

311
If you have point clouds in other format (`off`, `obj`, etc.), you can use `trimesh` to convert them into `ply`.
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326

```python
import trimesh

def to_ply(input_path, output_path, original_type):
    mesh = trimesh.load(input_path, file_type=original_type)  # read file
    mesh.export(output_path, file_type='ply')  # convert to ply
```

Examples:

```python
to_ply('./test.obj', './test.ply', 'obj')
```

327
More demos about single/multi-modality and indoor/outdoor 3D detection can be found in [demo](demo.md).
328

twang's avatar
twang committed
329
## High-level APIs for testing point clouds
zhangwenwei's avatar
zhangwenwei committed
330

twang's avatar
twang committed
331
### Synchronous interface
Ziyi Wu's avatar
Ziyi Wu committed
332

liyinhao's avatar
liyinhao committed
333
Here is an example of building the model and test given point clouds.
zhangwenwei's avatar
zhangwenwei committed
334
335

```python
336
from mmdet3d.apis import init_model, inference_detector
zhangwenwei's avatar
zhangwenwei committed
337

liyinhao's avatar
liyinhao committed
338
339
config_file = 'configs/votenet/votenet_8x8_scannet-3d-18class.py'
checkpoint_file = 'checkpoints/votenet_8x8_scannet-3d-18class_20200620_230238-2cea9c3a.pth'
zhangwenwei's avatar
zhangwenwei committed
340
341

# build the model from a config file and a checkpoint file
342
model = init_model(config_file, checkpoint_file, device='cuda:0')
zhangwenwei's avatar
zhangwenwei committed
343
344

# test a single image and show the results
liyinhao's avatar
liyinhao committed
345
346
347
348
point_cloud = 'test.bin'
result, data = inference_detector(model, point_cloud)
# visualize the results and save the results in 'results' folder
model.show_results(data, result, out_dir='results')
zhangwenwei's avatar
zhangwenwei committed
349
```