getting_started.md 11.1 KB
Newer Older
twang's avatar
twang committed
1
# Prerequisites
2

3
4
In this section we demonstrate how to prepare an environment with PyTorch.
MMDection3D works on Linux, Windows (experimental support) and macOS and requires the following packages:
zhangwenwei's avatar
zhangwenwei committed
5

twang's avatar
twang committed
6
- Python 3.6+
7
- PyTorch 1.6+
twang's avatar
twang committed
8
9
- CUDA 9.2+ (If you build PyTorch from source, CUDA 9.0 is also compatible)
- GCC 5+
10
11
- [MMEngine](https://mmengine.readthedocs.io/zh_CN/latest/#installation)
- [MMCV](https://mmcv.readthedocs.io/zh_CN/latest/#installation)
xiliu8006's avatar
xiliu8006 committed
12

13
14
```{note}
If you are experienced with PyTorch and have already installed it, just skip this part and jump to the [next section](#installation). Otherwise, you can follow these steps for the preparation.
Wenwei Zhang's avatar
Wenwei Zhang committed
15
16
```

17
**Step 0.** Download and install Miniconda from the [official website](https://docs.conda.io/en/latest/miniconda.html).
Wenwei Zhang's avatar
Wenwei Zhang committed
18

19
**Step 1.** Create a conda environment and activate it.
Wenwei Zhang's avatar
Wenwei Zhang committed
20

21
```shell
22
23
# We recommend to install python=3.8 since the waymo-open-dataset-tf-2-6-0 requires python>=3.7
# If you want to install python<3.7, make sure to install waymo-open-dataset-tf-2-x-0 (x<=4)
24
25
conda create --name openmmlab python=3.8 -y
conda activate openmmlab
Wenwei Zhang's avatar
Wenwei Zhang committed
26
27
```

28
**Step 2.** Install PyTorch following [official instructions](https://pytorch.org/get-started/locally/), e.g.
zhangwenwei's avatar
zhangwenwei committed
29

30
On GPU platforms:
zhangwenwei's avatar
zhangwenwei committed
31

Ziyi Wu's avatar
Ziyi Wu committed
32
```shell
33
conda install pytorch torchvision -c pytorch
xiliu8006's avatar
xiliu8006 committed
34
35
```

36
On CPU platforms:
xiliu8006's avatar
xiliu8006 committed
37

twang's avatar
twang committed
38
```shell
39
conda install pytorch torchvision cpuonly -c pytorch
twang's avatar
twang committed
40
```
zhangwenwei's avatar
zhangwenwei committed
41

42
# Installation
43

44
We recommend that users follow our best practices to install MMDetection3D. However, the whole process is highly customizable. See [Customize Installation](#customize-installation) section for more information.
45

46
## Best Practices
47

48
49
Assuming that you already have CUDA 11.0 installed, here is a full script for quick installation of MMDetection3D with conda.
Otherwise, you should refer to the step-by-step installation instructions in the next section.
zhangwenwei's avatar
zhangwenwei committed
50

twang's avatar
twang committed
51
```shell
52
pip install openmim
53
mim install mmengine
54
55
mim install 'mmcv>=2.0.0rc0'
mim install 'mmdet>=3.0.0rc0'
56
git clone https://github.com/open-mmlab/mmdetection3d.git -b dev-1.x
57
58
cd mmdetection3d
pip install -e .
xiliu8006's avatar
xiliu8006 committed
59
60
```

61
**Step 0.** Install [MMEngine](https://github.com/open-mmlab/mmengine) and [MMCV](https://github.com/open-mmlab/mmcv) using [MIM](https://github.com/open-mmlab/mim).
zhangwenwei's avatar
zhangwenwei committed
62
63

```shell
64
65
pip install -U openmim
mim install mmengine
66
mim install 'mmcv>=2.0.0rc0'
zhangwenwei's avatar
zhangwenwei committed
67
68
```

69
**Step 1.** Install [MMDetection](https://github.com/open-mmlab/mmdetection).
70
71

```shell
72
mim install 'mmdet>=3.0.0rc0'
73
74
```

75
Optionally, you could also build MMDetection from source in case you want to modify the code:
76
77

```shell
78
79
80
81
82
83
84
git clone https://github.com/open-mmlab/mmdetection.git -b dev-3.x
# "-b dev-3.x" means checkout to the `dev-3.x` branch.
cd mmdetection
pip install -v -e .
# "-v" means verbose, or more output
# "-e" means installing a project in editable mode,
# thus any local modifications made to the code will take effect without reinstallation.
85
86
```

87
**Step 2.** Clone the MMDetection3D repository.
zhangwenwei's avatar
Doc  
zhangwenwei committed
88

twang's avatar
twang committed
89
```shell
90
91
git clone https://github.com/open-mmlab/mmdetection3d.git -b dev-1.x
# "-b dev-1.x" means checkout to the `dev-1.x` branch.
twang's avatar
twang committed
92
93
cd mmdetection3d
```
zhangwenwei's avatar
zhangwenwei committed
94

95
**Step 3.** Install build requirements and then install MMDetection3D.
zhangwenwei's avatar
zhangwenwei committed
96

twang's avatar
twang committed
97
98
99
```shell
pip install -v -e .  # or "python setup.py develop"
```
zhangwenwei's avatar
zhangwenwei committed
100

twang's avatar
twang committed
101
Note:
zhangwenwei's avatar
Doc  
zhangwenwei committed
102

103
1. The git commit id will be written to the version number with step 4, e.g. 0.6.0+2e7045c. The version will also be saved in trained models.
104
   It is recommended that you run step d each time you pull some updates from github. If C++/CUDA codes are modified, then this step is compulsory.
zhangwenwei's avatar
Doc  
zhangwenwei committed
105

106
   > Important: Be sure to remove the `./build` folder if you reinstall mmdet with a different CUDA/PyTorch version.
zhangwenwei's avatar
zhangwenwei committed
107

108
109
110
111
112
   ```shell
   pip uninstall mmdet3d
   rm -rf ./build
   find . -name "*.so" | xargs rm
   ```
zhangwenwei's avatar
zhangwenwei committed
113

114
2. Following the above instructions, MMDetection3D is installed on `dev` mode, any local modifications made to the code will take effect without the need to reinstall it (unless you submit some commits and want to update the version number).
zhangwenwei's avatar
zhangwenwei committed
115

twang's avatar
twang committed
116
3. If you would like to use `opencv-python-headless` instead of `opencv-python`,
117
   you can install it before installing MMCV.
zhangwenwei's avatar
zhangwenwei committed
118

twang's avatar
twang committed
119
4. Some dependencies are optional. Simply running `pip install -v -e .` will only install the minimum runtime requirements. To use optional dependencies like `albumentations` and `imagecorruptions` either install them manually with `pip install -r requirements/optional.txt` or specify desired extras when calling `pip` (e.g. `pip install -v -e .[optional]`). Valid keys for the extras field are: `all`, `tests`, `build`, and `optional`.
zhangwenwei's avatar
zhangwenwei committed
120

VVsssssk's avatar
VVsssssk committed
121
122
123
124
125
126
127
128
129
130
131
132
133
   We have supported spconv2.0. If the user has installed spconv2.0, the code will use spconv2.0 first, which will take up less GPU memory than using the default mmcv spconv. Users can use the following commands to install spconv2.0:

   ```bash
   pip install cumm-cuxxx
   pip install spconv-cuxxx
   ```

   Where xxx is the CUDA version in the environment.

   For example, using CUDA 10.2, the command will be `pip install cumm-cu102 && pip install spconv-cu102`.

   Supported CUDA versions include 10.2, 11.1, 11.3, and 11.4. Users can also install it by building from the source. For more details please refer to [spconv v2.x](https://github.com/traveller59/spconv).

134
135
   We also support Minkowski Engine as a sparse convolution backend. If necessary please follow original [installation guide](https://github.com/NVIDIA/MinkowskiEngine#installation) or use `pip`:

136
137
138
139
   ```shell
   conda install openblas-devel -c anaconda
   pip install -U git+https://github.com/NVIDIA/MinkowskiEngine -v --no-deps --install-option="--blas_include_dirs=/opt/conda/include" --install-option="--blas=openblas"
   ```
140

twang's avatar
twang committed
141
5. The code can not be built for CPU only environment (where CUDA isn't available) for now.
zhangwenwei's avatar
zhangwenwei committed
142

143
## Verification
Wenwei Zhang's avatar
Wenwei Zhang committed
144

145
### Verify with point cloud demo
zhangwenwei's avatar
Doc  
zhangwenwei committed
146

147
We provide several demo scripts to test a single sample. Pre-trained models can be downloaded from [model zoo](model_zoo.md). To test a single-modality 3D detection on point cloud scenes:
zhangwenwei's avatar
Doc  
zhangwenwei committed
148
149

```shell
wuyuefeng's avatar
Demo  
wuyuefeng committed
150
python demo/pcd_demo.py ${PCD_FILE} ${CONFIG_FILE} ${CHECKPOINT_FILE} [--device ${GPU_ID}] [--score-thr ${SCORE_THR}] [--out-dir ${OUT_DIR}]
zhangwenwei's avatar
Doc  
zhangwenwei committed
151
152
153
154
155
```

Examples:

```shell
156
python demo/pcd_demo.py demo/data/kitti/000008.bin configs/second/second_hv-secfpn_8xb6-80e_kitti-3d-car.py checkpoints/second_hv-secfpn_8xb6-80e_kitti-3d-car_20200620_230238-393f000c.pth
zhangwenwei's avatar
zhangwenwei committed
157
```
158

yinchimaoliang's avatar
yinchimaoliang committed
159
If you want to input a `ply` file, you can use the following function and convert it to `bin` format. Then you can use the converted `bin` file to generate demo.
160
Note that you need to install `pandas` and `plyfile` before using this script. This function can also be used for data preprocessing for training `ply data`.
161

yinchimaoliang's avatar
yinchimaoliang committed
162
163
164
165
166
```python
import numpy as np
import pandas as pd
from plyfile import PlyData

167
def convert_ply(input_path, output_path):
yinchimaoliang's avatar
yinchimaoliang committed
168
169
170
171
172
173
174
175
176
177
    plydata = PlyData.read(input_path)  # read file
    data = plydata.elements[0].data  # read data
    data_pd = pd.DataFrame(data)  # convert to DataFrame
    data_np = np.zeros(data_pd.shape, dtype=np.float)  # initialize array to store data
    property_names = data[0].dtype.names  # read names of properties
    for i, name in enumerate(
            property_names):  # read data by property
        data_np[:, i] = data_pd[name]
    data_np.astype(np.float32).tofile(output_path)
```
178

yinchimaoliang's avatar
yinchimaoliang committed
179
Examples:
zhangwenwei's avatar
zhangwenwei committed
180

yinchimaoliang's avatar
yinchimaoliang committed
181
182
183
```python
convert_ply('./test.ply', './test.bin')
```
zhangwenwei's avatar
zhangwenwei committed
184

185
If you have point clouds in other format (`off`, `obj`, etc.), you can use `trimesh` to convert them into `ply`.
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200

```python
import trimesh

def to_ply(input_path, output_path, original_type):
    mesh = trimesh.load(input_path, file_type=original_type)  # read file
    mesh.export(output_path, file_type='ply')  # convert to ply
```

Examples:

```python
to_ply('./test.obj', './test.ply', 'obj')
```

201
More demos about single/multi-modality and indoor/outdoor 3D detection can be found in [demo](user_guides/inference.md).
202

203
## Customize Installation
zhangwenwei's avatar
zhangwenwei committed
204

205
### CUDA Versions
206

207
When installing PyTorch, you need to specify the version of CUDA. If you are not clear on which to choose, follow our recommendations:
Ziyi Wu's avatar
Ziyi Wu committed
208

209
210
- For Ampere-based NVIDIA GPUs, such as GeForce 30 series and NVIDIA A100, CUDA 11 is a must.
- For older NVIDIA GPUs, CUDA 11 is backward compatible, but CUDA 10.2 offers better compatibility and is more lightweight.
zhangwenwei's avatar
zhangwenwei committed
211

212
213
214
215
216
217
Please make sure the GPU driver satisfies the minimum version requirements. See [this table](https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#cuda-major-component-versions__table-cuda-toolkit-driver-versions) for more information.

```{note}
Installing CUDA runtime libraries is enough if you follow our best practices, because no CUDA code will be compiled locally. However if you hope to compile MMCV from source or develop other CUDA operators, you need to install the complete CUDA toolkit from NVIDIA's [website](https://developer.nvidia.com/cuda-downloads), and its version should match the CUDA version of PyTorch. i.e., the specified version of cudatoolkit in `conda install` command.
```

218
219
### Install MMEngine without MIM

220
To install MMEngine with pip instead of MIM, please follow [MMEngine installation guides](https://mmengine.readthedocs.io/en/latest/get_started/installation.html).
221
222
223
224
225
226
227

For example, you can install MMEngine by the following command.

```shell
pip install mmengine
```

228
229
230
231
232
233
### Install MMCV without MIM

MMCV contains C++ and CUDA extensions, thus depending on PyTorch in a complex way. MIM solves such dependencies automatically and makes the installation easier. However, it is not a must.

To install MMCV with pip instead of MIM, please follow [MMCV installation guides](https://mmcv.readthedocs.io/en/latest/get_started/installation.html). This requires manually specifying a find-url based on PyTorch version and its CUDA version.

234
For example, the following command install mmcv built for PyTorch 1.10.x and CUDA 11.3.
235
236

```shell
237
pip install mmcv -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.10/index.html
238
239
240
241
242
```

### Using MMDetection3D with Docker

We provide a [Dockerfile](https://github.com/open-mmlab/mmdetection3d/blob/master/docker/Dockerfile) to build an image.
zhangwenwei's avatar
zhangwenwei committed
243

244
245
246
247
```shell
# build an image with PyTorch 1.6, CUDA 10.1
docker build -t mmdetection3d -f docker/Dockerfile .
```
zhangwenwei's avatar
zhangwenwei committed
248

249
Run it with
zhangwenwei's avatar
zhangwenwei committed
250

251
252
```shell
docker run --gpus all --shm-size=8g -it -v {DATA_DIR}:/mmdetection3d/data mmdetection3d
zhangwenwei's avatar
zhangwenwei committed
253
```
254
255
256
257
258
259

### A from-scratch setup script

Here is a full script for setting up MMdetection3D with conda.

```shell
260
261
# We recommend to install python=3.8 since the waymo-open-dataset-tf-2-6-0 requires python>=3.7
# If you want to install python<3.7, make sure to install waymo-open-dataset-tf-2-x-0 (x<=4)
262
conda create -n open-mmlab python=3.8 -y
263
264
265
266
267
conda activate open-mmlab

# install latest PyTorch prebuilt with the default prebuilt CUDA version (usually the latest)
conda install -c pytorch pytorch torchvision -y

268
269
270
# install mmengine and mmcv
pip install openmim
mim install mmengine
271
mim install 'mmcv>=2.0.0rc0'
272
273

# install mmdetection
274
mim install 'mmdet>=3.0.0rc0'
275
276

# install mmdetection3d
277
git clone https://github.com/open-mmlab/mmdetection3d.git -b dev-1.x
278
cd mmdetection3d
279
pip install -e .
280
281
282
283
```

## Trouble shooting

284
If you have some issues during the installation, please first view the [FAQ](notes/faq.md) page.
285
You may [open an issue](https://github.com/open-mmlab/mmdetection3d/issues/new/choose) on GitHub if no solution is found.