get_started.md 11.9 KB
Newer Older
Xiang Xu's avatar
Xiang Xu committed
1
# Get Started
2

Xiang Xu's avatar
Xiang Xu committed
3
## Prerequisites
zhangwenwei's avatar
zhangwenwei committed
4

Xiang Xu's avatar
Xiang Xu committed
5
6
7
In this section, we demonstrate how to prepare an environment with PyTorch.

MMDetection3D works on Linux, Windows (experimental support) and macOS. It requires Python 3.7+, CUDA 9.2+, and PyTorch 1.6+.
xiliu8006's avatar
xiliu8006 committed
8

9
10
```{note}
If you are experienced with PyTorch and have already installed it, just skip this part and jump to the [next section](#installation). Otherwise, you can follow these steps for the preparation.
Wenwei Zhang's avatar
Wenwei Zhang committed
11
12
```

13
**Step 0.** Download and install Miniconda from the [official website](https://docs.conda.io/en/latest/miniconda.html).
Wenwei Zhang's avatar
Wenwei Zhang committed
14

15
**Step 1.** Create a conda environment and activate it.
Wenwei Zhang's avatar
Wenwei Zhang committed
16

17
```shell
18
19
conda create --name openmmlab python=3.8 -y
conda activate openmmlab
Wenwei Zhang's avatar
Wenwei Zhang committed
20
21
```

22
**Step 2.** Install PyTorch following [official instructions](https://pytorch.org/get-started/locally/), e.g.
zhangwenwei's avatar
zhangwenwei committed
23

24
On GPU platforms:
zhangwenwei's avatar
zhangwenwei committed
25

Ziyi Wu's avatar
Ziyi Wu committed
26
```shell
27
conda install pytorch torchvision -c pytorch
xiliu8006's avatar
xiliu8006 committed
28
29
```

30
On CPU platforms:
xiliu8006's avatar
xiliu8006 committed
31

twang's avatar
twang committed
32
```shell
33
conda install pytorch torchvision cpuonly -c pytorch
twang's avatar
twang committed
34
```
zhangwenwei's avatar
zhangwenwei committed
35

Xiang Xu's avatar
Xiang Xu committed
36
## Installation
37

38
We recommend that users follow our best practices to install MMDetection3D. However, the whole process is highly customizable. See [Customize Installation](#customize-installation) section for more information.
39

Xiang Xu's avatar
Xiang Xu committed
40
### Best Practices
41

Xiang Xu's avatar
Xiang Xu committed
42
**Step 0.** Install [MMEngine](https://github.com/open-mmlab/mmengine), [MMCV](https://github.com/open-mmlab/mmcv) and [MMDetection](https://github.com/open-mmlab/mmdetection) using [MIM](https://github.com/open-mmlab/mim).
zhangwenwei's avatar
zhangwenwei committed
43

twang's avatar
twang committed
44
```shell
45
pip install -U openmim
46
mim install mmengine
Xiang Xu's avatar
Xiang Xu committed
47
mim install 'mmcv>=2.0.0rc1'
48
mim install 'mmdet>=3.0.0rc0'
xiliu8006's avatar
xiliu8006 committed
49
50
```

Xiang Xu's avatar
Xiang Xu committed
51
**Note**: In MMCV-v2.x, `mmcv-full` is renamed to `mmcv`, if you want to install `mmcv` without CUDA ops, you can use `mim install "mmcv-lite>=2.0.0rc1"` to install the lite version.
zhangwenwei's avatar
zhangwenwei committed
52

Xiang Xu's avatar
Xiang Xu committed
53
**Step 1.** Install MMDetection3D.
zhangwenwei's avatar
zhangwenwei committed
54

Xiang Xu's avatar
Xiang Xu committed
55
Case a: If you develop and run mmdet3d directly, install it from source:
zhangwenwei's avatar
Doc  
zhangwenwei committed
56

twang's avatar
twang committed
57
```shell
58
59
git clone https://github.com/open-mmlab/mmdetection3d.git -b dev-1.x
# "-b dev-1.x" means checkout to the `dev-1.x` branch.
twang's avatar
twang committed
60
cd mmdetection3d
Xiang Xu's avatar
Xiang Xu committed
61
62
63
64
pip install -v -e .
# "-v" means verbose, or more output
# "-e" means installing a project in edtiable mode,
# thus any local modifications made to the code will take effect without reinstallation.
twang's avatar
twang committed
65
```
zhangwenwei's avatar
zhangwenwei committed
66

Xiang Xu's avatar
Xiang Xu committed
67
Case b: If you use mmdet3d as a dependency or third-party package, install it with MIM:
zhangwenwei's avatar
zhangwenwei committed
68

twang's avatar
twang committed
69
```shell
Xiang Xu's avatar
Xiang Xu committed
70
mim install "mmdet3d>=1.1.0rc0"
twang's avatar
twang committed
71
```
zhangwenwei's avatar
zhangwenwei committed
72

twang's avatar
twang committed
73
Note:
zhangwenwei's avatar
Doc  
zhangwenwei committed
74

Xiang Xu's avatar
Xiang Xu committed
75
1. If you would like to use `opencv-python-headless` instead of `opencv-python`,
76
   you can install it before installing MMCV.
zhangwenwei's avatar
zhangwenwei committed
77

Xiang Xu's avatar
Xiang Xu committed
78
2. Some dependencies are optional. Simply running `pip install -v -e .` will only install the minimum runtime requirements. To use optional dependencies like `albumentations` and `imagecorruptions` either install them manually with `pip install -r requirements/optional.txt` or specify desired extras when calling `pip` (e.g. `pip install -v -e .[optional]`). Valid keys for the extras field are: `all`, `tests`, `build`, and `optional`.
zhangwenwei's avatar
zhangwenwei committed
79

80
   We have supported `spconv 2.0`. If the user has installed `spconv 2.0`, the code will use `spconv 2.0` first, which will take up less GPU memory than using the default `mmcv spconv`. Users can use the following commands to install `spconv 2.0`:
VVsssssk's avatar
VVsssssk committed
81

Xiang Xu's avatar
Xiang Xu committed
82
   ```shell
VVsssssk's avatar
VVsssssk committed
83
84
85
86
   pip install cumm-cuxxx
   pip install spconv-cuxxx
   ```

87
   Where `xxx` is the CUDA version in the environment.
VVsssssk's avatar
VVsssssk committed
88
89
90
91
92

   For example, using CUDA 10.2, the command will be `pip install cumm-cu102 && pip install spconv-cu102`.

   Supported CUDA versions include 10.2, 11.1, 11.3, and 11.4. Users can also install it by building from the source. For more details please refer to [spconv v2.x](https://github.com/traveller59/spconv).

93
   We also support `Minkowski Engine` as a sparse convolution backend. If necessary please follow original [installation guide](https://github.com/NVIDIA/MinkowskiEngine#installation) or use `pip` to install it:
94

95
96
97
98
   ```shell
   conda install openblas-devel -c anaconda
   pip install -U git+https://github.com/NVIDIA/MinkowskiEngine -v --no-deps --install-option="--blas_include_dirs=/opt/conda/include" --install-option="--blas=openblas"
   ```
99

100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
   We also support `Torchsparse` as a sparse convolution backend. If necessary please follow original [installation guide](https://github.com/mit-han-lab/torchsparse#installation) or use `pip` to install it:

   ```shell
   sudo apt-get install libsparsehash-dev
   pip install --upgrade git+https://github.com/mit-han-lab/torchsparse.git@v1.4.0
   ```

   or omit sudo install by following command:

   ```shell
   conda install -c bioconda sparsehash
   export CPLUS_INCLUDE_PATH=CPLUS_INCLUDE_PATH:${YOUR_CONDA_ENVS_DIR}/include
   pip install --upgrade git+https://github.com/mit-han-lab/torchsparse.git@v1.4.0
   ```

Xiang Xu's avatar
Xiang Xu committed
115
3. The code can not be built for CPU only environment (where CUDA isn't available) for now.
zhangwenwei's avatar
zhangwenwei committed
116

Xiang Xu's avatar
Xiang Xu committed
117
### Verify the Installation
Wenwei Zhang's avatar
Wenwei Zhang committed
118

Xiang Xu's avatar
Xiang Xu committed
119
To verify whether MMDetection3D is installed correctly, we provide some sample codes to run an inference demo.
zhangwenwei's avatar
Doc  
zhangwenwei committed
120

Xiang Xu's avatar
Xiang Xu committed
121
**Step 1.** We need to download config and checkpoint files.
zhangwenwei's avatar
Doc  
zhangwenwei committed
122
123

```shell
Xiang Xu's avatar
Xiang Xu committed
124
mim download mmdet3d --config pointpillars_hv_secfpn_8xb6-160e_kitti-3d-car --dest .
zhangwenwei's avatar
Doc  
zhangwenwei committed
125
126
```

Xiang Xu's avatar
Xiang Xu committed
127
128
129
130
131
The downloading will take several seconds or more, depending on your network environment. When it is done, you will find two files `pointpillars_hv_secfpn_8xb6-160e_kitti-3d-car.py` and `hv_pointpillars_secfpn_6x8_160e_kitti-3d-car_20220331_134606-d42d15ed.pth` in your current folder.

**Step 2.** Verify the inference demo.

Case a: If you install MMDetection3D from source, just run the following command.
zhangwenwei's avatar
Doc  
zhangwenwei committed
132
133

```shell
Xiang Xu's avatar
Xiang Xu committed
134
python demo/pcd_demo.py demo/data/kitti/000008.bin pointpillars_hv_secfpn_8xb6-160e_kitti-3d-car.py hv_pointpillars_secfpn_6x8_160e_kitti-3d-car_20220331_134606-d42d15ed.pth --show
zhangwenwei's avatar
zhangwenwei committed
135
```
136

Xiang Xu's avatar
Xiang Xu committed
137
138
139
140
You will see a visualizer interface with point cloud, where bounding boxes are plotted on cars.

**Note**:

141
If you want to input a `.ply` file, you can use the following function and convert it to `.bin` format. Then you can use the converted `.bin` file to run demo.
142
Note that you need to install `pandas` and `plyfile` before using this script. This function can also be used for data preprocessing for training `ply data`.
143

yinchimaoliang's avatar
yinchimaoliang committed
144
145
146
147
148
```python
import numpy as np
import pandas as pd
from plyfile import PlyData

149
def convert_ply(input_path, output_path):
yinchimaoliang's avatar
yinchimaoliang committed
150
151
152
153
154
155
156
157
158
159
    plydata = PlyData.read(input_path)  # read file
    data = plydata.elements[0].data  # read data
    data_pd = pd.DataFrame(data)  # convert to DataFrame
    data_np = np.zeros(data_pd.shape, dtype=np.float)  # initialize array to store data
    property_names = data[0].dtype.names  # read names of properties
    for i, name in enumerate(
            property_names):  # read data by property
        data_np[:, i] = data_pd[name]
    data_np.astype(np.float32).tofile(output_path)
```
160

yinchimaoliang's avatar
yinchimaoliang committed
161
Examples:
zhangwenwei's avatar
zhangwenwei committed
162

yinchimaoliang's avatar
yinchimaoliang committed
163
164
165
```python
convert_ply('./test.ply', './test.bin')
```
zhangwenwei's avatar
zhangwenwei committed
166

167
If you have point clouds in other format (`.off`, `.obj`, etc.), you can use `trimesh` to convert them into `.ply`.
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182

```python
import trimesh

def to_ply(input_path, output_path, original_type):
    mesh = trimesh.load(input_path, file_type=original_type)  # read file
    mesh.export(output_path, file_type='ply')  # convert to ply
```

Examples:

```python
to_ply('./test.obj', './test.ply', 'obj')
```

Xiang Xu's avatar
Xiang Xu committed
183
184
185
186
187
188
189
190
191
192
193
194
Case b: If you install MMDetection3D with MIM, open your python interpreter and copy&paste the following codes.

```python
from mmdet3d.apis import init_model, inference_detector

config_file = 'pointpillars_hv_secfpn_8xb6-160e_kitti-3d-car.py'
checkpoint_file = 'hv_pointpillars_secfpn_6x8_160e_kitti-3d-car_20220331_134606-d42d15ed.pth'
model = init_model(config_file, checkpoint_file)
inference_detector(model, 'demo/data/kitti/000008.bin')
```

You will see a list of `Det3DDataSample`, and the predictions are in the `pred_instances_3d`, indicating the detected bounding boxes, labels, and scores.
195

Xiang Xu's avatar
Xiang Xu committed
196
### Customize Installation
zhangwenwei's avatar
zhangwenwei committed
197

Xiang Xu's avatar
Xiang Xu committed
198
#### CUDA Versions
199

200
When installing PyTorch, you need to specify the version of CUDA. If you are not clear on which to choose, follow our recommendations:
Ziyi Wu's avatar
Ziyi Wu committed
201

202
203
- For Ampere-based NVIDIA GPUs, such as GeForce 30 series and NVIDIA A100, CUDA 11 is a must.
- For older NVIDIA GPUs, CUDA 11 is backward compatible, but CUDA 10.2 offers better compatibility and is more lightweight.
zhangwenwei's avatar
zhangwenwei committed
204

205
206
207
208
209
210
Please make sure the GPU driver satisfies the minimum version requirements. See [this table](https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#cuda-major-component-versions__table-cuda-toolkit-driver-versions) for more information.

```{note}
Installing CUDA runtime libraries is enough if you follow our best practices, because no CUDA code will be compiled locally. However if you hope to compile MMCV from source or develop other CUDA operators, you need to install the complete CUDA toolkit from NVIDIA's [website](https://developer.nvidia.com/cuda-downloads), and its version should match the CUDA version of PyTorch. i.e., the specified version of cudatoolkit in `conda install` command.
```

Xiang Xu's avatar
Xiang Xu committed
211
#### Install MMEngine without MIM
212

213
To install MMEngine with pip instead of MIM, please follow [MMEngine installation guides](https://mmengine.readthedocs.io/en/latest/get_started/installation.html).
214

215
For example, you can install MMEngine by the following command:
216
217
218
219
220

```shell
pip install mmengine
```

Xiang Xu's avatar
Xiang Xu committed
221
#### Install MMCV without MIM
222
223
224

MMCV contains C++ and CUDA extensions, thus depending on PyTorch in a complex way. MIM solves such dependencies automatically and makes the installation easier. However, it is not a must.

225
To install MMCV with pip instead of MIM, please follow [MMCV installation guides](https://mmcv.readthedocs.io/en/2.x/get_started/installation.html). This requires manually specifying a find-url based on PyTorch version and its CUDA version.
226

Xiang Xu's avatar
Xiang Xu committed
227
For example, the following command install MMCV built for PyTorch 1.12.x and CUDA 11.6:
228
229

```shell
Xiang Xu's avatar
Xiang Xu committed
230
pip install "mmcv>=2.0.0rc1" -f https://download.openmmlab.com/mmcv/dist/cu116/torch1.12.0/index.html
231
232
```

Xiang Xu's avatar
Xiang Xu committed
233
#### Install on Google Colab
234

Xiang Xu's avatar
Xiang Xu committed
235
236
237
[Google Colab](https://colab.research.google.com/) usually has PyTorch installed, thus we only need to install MMEngine, MMCV, MMDetection, and MMDetection3D with the following commands.

**Step 1.** Install [MMEngine](https://github.com/open-mmlab/mmengine), [MMCV](https://github.com/open-mmlab/mmcv) and [MMDetection](https://github.com/open-mmlab/mmdetection) using [MIM](https://github.com/open-mmlab/mim).
zhangwenwei's avatar
zhangwenwei committed
238

239
```shell
Xiang Xu's avatar
Xiang Xu committed
240
241
242
243
!pip3 install openmim
!mim install mmengine
!mim install "mmcv>=2.0.0rc1,<2.1.0"
!mim install "mmdet>=3.0.0rc0,<3.1.0"
244
```
zhangwenwei's avatar
zhangwenwei committed
245

Xiang Xu's avatar
Xiang Xu committed
246
**Step 2.** Install MMDetection3D from source.
zhangwenwei's avatar
zhangwenwei committed
247

248
```shell
Xiang Xu's avatar
Xiang Xu committed
249
250
251
!git clone https://github.com/open-mmlab/mmdetection3d.git -b dev-1.x
%cd mmdetection3d
!pip install -e .
zhangwenwei's avatar
zhangwenwei committed
252
```
253

Xiang Xu's avatar
Xiang Xu committed
254
**Step 3.** Verification.
255

Xiang Xu's avatar
Xiang Xu committed
256
257
258
259
260
```python
import mmdet3d
print(mmdet3d.__version__)
# Example output: 1.1.0rc0, or an another version.
```
261

Xiang Xu's avatar
Xiang Xu committed
262
263
264
```{note}
Within Jupyter, the exclamation mark `!` is used to call external executables and `%cd` is a [magic command](https://ipython.readthedocs.io/en/stable/interactive/magics.html#magic-cd) to change the current working directory of Python.
```
265

Xiang Xu's avatar
Xiang Xu committed
266
#### Using MMDetection3D with Docker
267

Xiang Xu's avatar
Xiang Xu committed
268
We provide a [Dockerfile](https://github.com/open-mmlab/mmdetection3d/blob/dev-1.x/docker/Dockerfile) to build an image. Ensure that your [docker version](https://docs.docker.com/engine/install/) >= 19.03.
269

Xiang Xu's avatar
Xiang Xu committed
270
```shell
271
# build an image with PyTorch 1.9, CUDA 11.1
Xiang Xu's avatar
Xiang Xu committed
272
273
274
# If you prefer other versions, just modified the Dockerfile
docker build -t mmdetection3d docker/
```
275

Xiang Xu's avatar
Xiang Xu committed
276
277
278
279
Run it with:

```shell
docker run --gpus all --shm-size=8g -it -v {DATA_DIR}:/mmdetection3d/data mmdetection3d
280
281
```

Xiang Xu's avatar
Xiang Xu committed
282
### Troubleshooting
283

284
If you have some issues during the installation, please first view the [FAQ](notes/faq.md) page.
285
You may [open an issue](https://github.com/open-mmlab/mmdetection3d/issues/new/choose) on GitHub if no solution is found.
Xiang Xu's avatar
Xiang Xu committed
286
287
288
289
290
291
292
293
294
295

### Use Multiple Versions of MMDetection3D in Development

Training and testing scripts have already been modified in `PYTHONPATH` in order to make sure the scripts are using their own versions of MMDetection3D.

To install the default version of MMDetection3D in your environment, you can exclude the following code in the related scripts:

```shell
PYTHONPATH="$(dirname $0)/..":$PYTHONPATH
```