README.md 11.7 KB
Newer Older
yan.yan's avatar
yan.yan committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
<!--
 Copyright 2021 Yan Yan
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at
 
     http://www.apache.org/licenses/LICENSE-2.0
 
 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
-->
16
17
18
[pypi-ver-cpu]: https://img.shields.io/pypi/v/spconv
[pypi-ver-114]: https://img.shields.io/pypi/v/spconv-cu114
[pypi-ver-111]: https://img.shields.io/pypi/v/spconv-cu111
yan.yan's avatar
yan.yan committed
19
[pypi-ver-117]: https://img.shields.io/pypi/v/spconv-cu117
20
21
[pypi-ver-116]: https://img.shields.io/pypi/v/spconv-cu116
[pypi-ver-118]: https://img.shields.io/pypi/v/spconv-cu118
yan.yan's avatar
yan.yan committed
22

23
[pypi-ver-113]: https://img.shields.io/pypi/v/spconv-cu113
yan.yan's avatar
yan.yan committed
24
[pypi-ver-120]: https://img.shields.io/pypi/v/spconv-cu120
25
26
[pypi-ver-102]: https://img.shields.io/pypi/v/spconv-cu102

yan.yan's avatar
yan.yan committed
27
28
[pypi-url-102]: https://pypi.org/project/spconv-cu102/
[pypi-download-102]: https://img.shields.io/pypi/dm/spconv-cu102
29
30
31
32
33
34
[pypi-url-111]: https://pypi.org/project/spconv-cu111/
[pypi-download-111]: https://img.shields.io/pypi/dm/spconv-cu111
[pypi-url-113]: https://pypi.org/project/spconv-cu113/
[pypi-download-113]: https://img.shields.io/pypi/dm/spconv-cu113
[pypi-url-114]: https://pypi.org/project/spconv-cu114/
[pypi-download-114]: https://img.shields.io/pypi/dm/spconv-cu114
yan.yan's avatar
yan.yan committed
35
36
[pypi-url-117]: https://pypi.org/project/spconv-cu117/
[pypi-download-117]: https://img.shields.io/pypi/dm/spconv-cu117
yan.yan's avatar
yan.yan committed
37
38
[pypi-url-120]: https://pypi.org/project/spconv-cu120/
[pypi-download-120]: https://img.shields.io/pypi/dm/spconv-cu120
39
40
[pypi-url-cpu]: https://pypi.org/project/spconv/
[pypi-download-cpu]: https://img.shields.io/pypi/dm/spconv
41
42
43
[pypi-url-118]: https://pypi.org/project/spconv-cu118/
[pypi-download-118]: https://img.shields.io/pypi/dm/spconv-cu118

yan.yan's avatar
yan.yan committed
44
45
[pypi-url-116]: https://pypi.org/project/spconv-cu116/
[pypi-download-116]: https://img.shields.io/pypi/dm/spconv-cu116
yan.yan's avatar
yan.yan committed
46

yan.yan's avatar
v2.1  
yan.yan committed
47
# SpConv: Spatially Sparse Convolution Library
48
[![Build Status](https://github.com/traveller59/spconv/workflows/build/badge.svg)](https://github.com/traveller59/spconv/actions?query=workflow%3Abuild) 
49
![pypi versions](https://img.shields.io/pypi/pyversions/spconv-cu117)
50

traveller59's avatar
traveller59 committed
51

52
|                | PyPI   | Install  |Downloads  |
yan.yan's avatar
yan.yan committed
53
54
| -------------- |:---------------------:| ---------------------:| ---------------------:| 
| CPU (Linux Only) | [![PyPI Version][pypi-ver-cpu]][pypi-url-cpu] | ```pip install spconv``` | [![pypi monthly download][pypi-download-cpu]][pypi-url-cpu] | 
yan.yan's avatar
yan.yan committed
55
| CUDA 10.2 | [![PyPI Version][pypi-ver-102]][pypi-url-102] | ```pip install spconv-cu102```| [![pypi monthly download][pypi-download-102]][pypi-url-102]| 
yan.yan's avatar
yan.yan committed
56
| CUDA 11.3 | [![PyPI Version][pypi-ver-113]][pypi-url-113] | ```pip install spconv-cu113```| [![pypi monthly download][pypi-download-113]][pypi-url-113]| 
yan.yan's avatar
yan.yan committed
57
| CUDA 11.4 | [![PyPI Version][pypi-ver-114]][pypi-url-114] | ```pip install spconv-cu114```| [![pypi monthly download][pypi-download-114]][pypi-url-114]|
58
| CUDA 11.6 | [![PyPI Version][pypi-ver-116]][pypi-url-116] | ```pip install spconv-cu116```| [![pypi monthly download][pypi-download-116]][pypi-url-116]|
yan.yan's avatar
yan.yan committed
59
| CUDA 11.7 | [![PyPI Version][pypi-ver-117]][pypi-url-117] | ```pip install spconv-cu117```| [![pypi monthly download][pypi-download-117]][pypi-url-117]| 
yan.yan's avatar
yan.yan committed
60
61
62
| CUDA 11.8* | [![PyPI Version][pypi-ver-118]][pypi-url-118] | ```pip install spconv-cu118```| [![pypi monthly download][pypi-download-118]][pypi-url-118]| 

*: sm_89 and sm_90 is added in CUDA 11.8. If you use RTX 4090 or H100, you should use this version.
63

yan.yan's avatar
yan.yan committed
64
<!-- | CUDA 12.0 | [![PyPI Version][pypi-ver-120]][pypi-url-120] | ```pip install spconv-cu120```| [![pypi monthly download][pypi-download-120]][pypi-url-120]| -->
tusimple's avatar
tusimple committed
65

66
```spconv``` is a project that provide heavily-optimized sparse convolution implementation with tensor core support. check [benchmark](docs/BENCHMARK.md) to see how fast spconv 2.x runs.
67

yan.yan's avatar
v2.1  
yan.yan committed
68
[Spconv 1.x code](https://github.com/traveller59/spconv/tree/v1.2.1). We won't provide any support for spconv 1.x since it's deprecated. use spconv 2.x if possible. <!--remove this message in spconv 2.2-->
FindDefinition's avatar
FindDefinition committed
69

yan.yan's avatar
yan.yan committed
70
71
Check [spconv 2.x algorithm introduction](docs/spconv2_algo.pdf) to understand sparse convolution algorithm in spconv 2.x!

yan.yan's avatar
yan.yan committed
72
73
74
75
## WARNING

Use spconv >= cu114 if possible. cuda 11.4 can compile greatly faster kernel in some situation.

yan.yan's avatar
yan.yan committed
76
77
Update Spconv: you **MUST UNINSTALL** all spconv/cumm/spconv-cuxxx/cumm-cuxxx first, use ```pip list | grep spconv``` and ```pip list | grep cumm``` to check all installed package. then use pip to install new spconv.

yan.yan's avatar
yan.yan committed
78
## NEWS
FindDefinition's avatar
FindDefinition committed
79

yan.yan's avatar
yan.yan committed
80
81
* spconv 2.3: int8 quantization support. see docs and examples for more details.

yan.yan's avatar
yan.yan committed
82
* spconv 2.2: ampere feature support (by [EvernightAurora](https://github.com/EvernightAurora)), pure c++ code generation, nvrtc, drop python 3.6
FindDefinition's avatar
FindDefinition committed
83

yan.yan's avatar
yan.yan committed
84
85
## Spconv 2.2 vs Spconv 2.1

yan.yan's avatar
yan.yan committed
86
* faster fp16 conv kernels (~5-30%) in ampere GPUs (tested in RTX 3090)
yan.yan's avatar
yan.yan committed
87
* greatly faster int8 conv kernels (~1.2x-2.7x) in ampere GPUs (tested in RTX 3090)
yan.yan's avatar
yan.yan committed
88
* drop python 3.6 support
yan.yan's avatar
yan.yan committed
89
90
91
* nvrtc support: kernel in old GPUs will be compiled in runtime.
* [libspconv](docs/PURE_CPP_BUILD.md): pure c++ build of all spconv ops. see [example](example/libspconv/run_build.sh)
* tf32 kernels, faster fp32 training, disabled by default. set ```import spconv as spconv_core; spconv_core.constants.SPCONV_ALLOW_TF32 = True``` to enable them.
yan.yan's avatar
yan.yan committed
92
* all weights are KRSC layout, some old model can't be loaded anymore.
yan.yan's avatar
yan.yan committed
93
94


yan.yan's avatar
v2.1  
yan.yan committed
95
## Spconv 2.1 vs Spconv 1.x
traveller59's avatar
traveller59 committed
96

yan.yan's avatar
yan.yan committed
97
* spconv now can be installed by **pip**. see install section in readme for more details. Users don't need to build manually anymore!
yan.yan's avatar
v2.1  
yan.yan committed
98
99
100
101
* Microsoft Windows support (only windows 10 has been tested).
* fp32 (not tf32) training/inference speed is increased (+50~80%)
* fp16 training/inference speed is greatly increased when your layer support tensor core (channel size must be multiple of 8).
* int8 op is ready, but we still need some time to figure out how to run int8 in pytorch.
102
* [doesn't depend on pytorch binary](docs/FAQ.md#What-does-no-dependency-on-pytorch-mean), but you may need at least pytorch >= 1.5.0 to run spconv 2.x.
yan.yan's avatar
yan.yan committed
103
* since spconv 2.x doesn't depend on pytorch binary (never in future), it's impossible to support torch.jit/libtorch inference.
traveller59's avatar
traveller59 committed
104

yan.yan's avatar
yan.yan committed
105
106
107
108
## Usage

Firstly you need to use ```import spconv.pytorch as spconv``` in spconv 2.x.

yan.yan's avatar
v2.1  
yan.yan committed
109
Then see [this](docs/USAGE.md).
yan.yan's avatar
yan.yan committed
110

yan.yan's avatar
v2.1  
yan.yan committed
111
Don't forget to check [performance guide](docs/PERFORMANCE_GUIDE.md).
traveller59's avatar
traveller59 committed
112

yan.yan's avatar
yan.yan committed
113
114
115
116
### Common Solution for Some Bugs

see [common problems](docs/COMMON_PROBLEMS.md).

yan.yan's avatar
yan.yan committed
117
## Install
traveller59's avatar
traveller59 committed
118

yan.yan's avatar
yan.yan committed
119
You need to install python >= 3.7 first to use spconv 2.x.
traveller59's avatar
traveller59 committed
120

yan.yan's avatar
yan.yan committed
121
You need to install CUDA toolkit first before using prebuilt binaries or build from source.
traveller59's avatar
traveller59 committed
122

yan.yan's avatar
yan.yan committed
123
You need at least CUDA 11.0 to build and run spconv 2.x. We won't offer any support for CUDA < 11.0.
traveller59's avatar
traveller59 committed
124

yan.yan's avatar
yan.yan committed
125
### Prebuilt
traveller59's avatar
traveller59 committed
126

yan.yan's avatar
yan.yan committed
127
We offer python 3.7-3.11 and cuda 10.2/11.3/11.4/11.7/12.0 prebuilt binaries for linux (manylinux).
yan.yan's avatar
yan.yan committed
128

yan.yan's avatar
yan.yan committed
129
We offer python 3.7-3.11 and cuda 10.2/11.4/11.7/12.0 prebuilt binaries for windows 10/11.
traveller59's avatar
traveller59 committed
130

yan.yan's avatar
yan.yan committed
131
For Linux users, you need to install pip >= 20.3 first to install prebuilt.
traveller59's avatar
traveller59 committed
132

yan.yan's avatar
yan.yan committed
133
134
**WARNING**: spconv-cu117 may require CUDA Driver >= 515.

yan.yan's avatar
v2.1  
yan.yan committed
135
136
137
138
```pip install spconv``` for CPU only (**Linux Only**). you should only use this for debug usage, the performance isn't optimized due to manylinux limit (no omp support).

```pip install spconv-cu102``` for CUDA 10.2

yan.yan's avatar
yan.yan committed
139
```pip install spconv-cu113``` for CUDA 11.3 (**Linux Only**)
yan.yan's avatar
v2.1  
yan.yan committed
140

yan.yan's avatar
yan.yan committed
141
```pip install spconv-cu114``` for CUDA 11.4
traveller59's avatar
traveller59 committed
142

yan.yan's avatar
yan.yan committed
143
144
```pip install spconv-cu117``` for CUDA 11.7

yan.yan's avatar
yan.yan committed
145
```pip install spconv-cu120``` for CUDA 12.0
146

yan.yan's avatar
yan.yan committed
147
**NOTE** It's safe to have different **minor** cuda version between system and conda (pytorch) in **CUDA >= 11.0** because of [CUDA Minor Version Compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/#minor-version-compatibility). For example, you can use spconv-cu114 with anaconda version of pytorch cuda 11.1 in a OS with CUDA 11.2 installed.
yan.yan's avatar
v2.1  
yan.yan committed
148

149
**NOTE** In Linux, you can install spconv-cuxxx without install CUDA to system! only suitable NVIDIA driver is required. for CUDA 11, we need driver >= 450.82. You may need newer driver if you use newer CUDA. for cuda 11.8, you need to have driver >= 520 installed.
yan.yan's avatar
v2.1  
yan.yan committed
150

151
152
153
154
#### Prebuilt GPU Support Matrix

See [this page](https://arnon.dk/matching-sm-architectures-arch-and-gencode-for-various-nvidia-cards/) to check supported GPU names by arch.

yan.yan's avatar
yan.yan committed
155
156
If you use a GPU architecture that isn't compiled in prebuilt, spconv will use NVRTC to compile a slightly slower kernel.

157
158
| CUDA version | GPU Arch List  |
| -------------- |:---------------------:|
159
160
| 11.1~11.7       | 52,60,61,70,75,80,86     | 
| 11.8+       | 60,70,75,80,86,89,90     | 
161

yan.yan's avatar
v2.1  
yan.yan committed
162
163
164
165
### Build from source for development (JIT, recommend)

The c++ code will be built automatically when you change c++ code in project.

166
For NVIDIA Embedded Platforms, you need to specify cuda arch before build: ```export CUMM_CUDA_ARCH_LIST="7.2"``` for xavier, ```export CUMM_CUDA_ARCH_LIST="6.2"``` for TX2, ```export CUMM_CUDA_ARCH_LIST="8.7"``` for orin.
yan.yan's avatar
v2.1  
yan.yan committed
167

168
169
You need to remove ```cumm``` in ```requires``` section in pyproject.toml after install editable ```cumm``` and before install spconv due to pyproject limit (can't find editable installed ```cumm```).

170
171
You need to ensure ```pip list | grep spconv``` and ```pip list | grep cumm``` show nothing before install editable spconv/cumm.

yan.yan's avatar
v2.1  
yan.yan committed
172
#### Linux
173

yan.yan's avatar
v2.1  
yan.yan committed
174
175
176
177
178
179
180
181
182
183
184
185
186
187
0. uninstall spconv and cumm installed by pip
1. install build-essential, install CUDA
2. ```git clone https://github.com/FindDefinition/cumm```, ```cd ./cumm```, ```pip install -e .```
3. ```git clone https://github.com/traveller59/spconv```, ```cd ./spconv```, ```pip install -e .```
4. in python, ```import spconv``` and wait for build finish.

#### Windows
0. uninstall spconv and cumm installed by pip
1. install visual studio 2019 or newer. make sure C++ development component is installed. install CUDA
2. set [powershell script execution policy](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_execution_policies?view=powershell-7.1)
3. start a new powershell, run ```tools/msvc_setup.ps1```
4. ```git clone https://github.com/FindDefinition/cumm```, ```cd ./cumm```, ```pip install -e .```
5. ```git clone https://github.com/traveller59/spconv```, ```cd ./spconv```, ```pip install -e .```
6. in python, ```import spconv``` and wait for build finish.
yan.yan's avatar
yan.yan committed
188

yan.yan's avatar
v2.1  
yan.yan committed
189
### Build wheel from source (not recommend, this is done in CI.)
traveller59's avatar
traveller59 committed
190

yan.yan's avatar
yan.yan committed
191
You need to rebuild ```cumm``` first if you are build along a CUDA version that not provided in prebuilts.
traveller59's avatar
traveller59 committed
192

yan.yan's avatar
yan.yan committed
193
#### Linux
traveller59's avatar
traveller59 committed
194

yan.yan's avatar
yan.yan committed
195
196
1. install build-essential, install CUDA
2. run ```export SPCONV_DISABLE_JIT="1"```
yan.yan's avatar
v2.1  
yan.yan committed
197
198
3. run ```pip install pccm cumm wheel```
4. run ```python setup.py bdist_wheel```+```pip install dists/xxx.whl```
traveller59's avatar
traveller59 committed
199

yan.yan's avatar
v2.1  
yan.yan committed
200
#### Windows
traveller59's avatar
traveller59 committed
201

yan.yan's avatar
v2.1  
yan.yan committed
202
1. install visual studio 2019 or newer. make sure C++ development component is installed. install CUDA
yan.yan's avatar
yan.yan committed
203
204
205
2. set [powershell script execution policy](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_execution_policies?view=powershell-7.1)
3. start a new powershell, run ```tools/msvc_setup.ps1```
4. run ```$Env:SPCONV_DISABLE_JIT = "1"```
yan.yan's avatar
v2.1  
yan.yan committed
206
207
5. run ```pip install pccm cumm wheel```
6. run ```python setup.py bdist_wheel```+```pip install dists/xxx.whl```
traveller59's avatar
traveller59 committed
208

209
210
211
212
213
214
215
216
217
218
219
220
## Citation

If you find this project useful in your research, please consider cite:

```latex
@misc{spconv2022,
    title={Spconv: Spatially Sparse Convolution Library},
    author={Spconv Contributors},
    howpublished = {\url{https://github.com/traveller59/spconv}},
    year={2022}
}
```
yan.yan's avatar
yan.yan committed
221
## Contributers
222

yan.yan's avatar
yan.yan committed
223
* [EvernightAurora](https://github.com/EvernightAurora): add ampere feature.
yan.yan's avatar
yan.yan committed
224

yan.yan's avatar
yan.yan committed
225
## Note
traveller59's avatar
traveller59 committed
226

yan.yan's avatar
yan.yan committed
227
The work is done when the author is an employee at [Tusimple](https://www.tusimple.com/).
traveller59's avatar
traveller59 committed
228

yan.yan's avatar
yan.yan committed
229
## LICENSE
traveller59's avatar
traveller59 committed
230

FindDefinition's avatar
FindDefinition committed
231
Apache 2.0