README.md 6.88 KB
Newer Older
pangjm's avatar
pangjm committed
1

WXinlong's avatar
WXinlong committed
2
# SOLO: Segmenting Objects by Locations
Kai Chen's avatar
Kai Chen committed
3

WXinlong's avatar
WXinlong committed
4
This project hosts the code for implementing the SOLO algorithms for instance segmentation.
Dahua Lin's avatar
Dahua Lin committed
5

WXinlong's avatar
WXinlong committed
6
> [**SOLO: Segmenting Objects by Locations**](https://arxiv.org/abs/1912.04488),            
Xinlong Wang's avatar
Xinlong Wang committed
7
8
> Xinlong Wang, Tao Kong, Chunhua Shen, Yuning Jiang, Lei Li    
> In: Proc. European Conference on Computer Vision (ECCV), 2020  
WXinlong's avatar
WXinlong committed
9
> *arXiv preprint ([arXiv 1912.04488](https://arxiv.org/abs/1912.04488))*   
Kai Chen's avatar
Kai Chen committed
10

Dahua Lin's avatar
Dahua Lin committed
11

Xinlong Wang's avatar
Xinlong Wang committed
12
> [**SOLOv2: Dynamic and Fast Instance Segmentation**](https://arxiv.org/abs/2003.10152),            
Xinlong Wang's avatar
Xinlong Wang committed
13
14
> Xinlong Wang, Rufeng Zhang, Tao Kong, Lei Li, Chunhua Shen     
> In: Proc. Advances in Neural Information Processing Systems (NeurIPS), 2020  
WXinlong's avatar
WXinlong committed
15
> *arXiv preprint ([arXiv 2003.10152](https://arxiv.org/abs/2003.10152))*  
Kai Chen's avatar
Kai Chen committed
16

WXinlong's avatar
WXinlong committed
17
More code and models will be released soon. Stay tuned.
Dahua Lin's avatar
Dahua Lin committed
18

Kai Chen's avatar
Kai Chen committed
19

WXinlong's avatar
WXinlong committed
20
21
22
## Highlights
- **Totally box-free:**  SOLO is totally box-free thus not being restricted by (anchor) box locations and scales, and naturally benefits from the inherent advantages of FCNs.
- **Direct instance segmentation:** Our method takes an image as input, directly outputs instance masks and corresponding class probabilities, in a fully convolutional, box-free and grouping-free paradigm.
WXinlong's avatar
WXinlong committed
23
- **High-quality mask prediction:** SOLOv2 is able to predict fine and detailed masks, especially at object boundaries.
WXinlong's avatar
WXinlong committed
24
- **State-of-the-art performance:** Our best single model based on ResNet-101 and deformable convolutions achieves **41.7%** in AP on COCO test-dev (without multi-scale testing). A light-weight version of SOLOv2 executes at **31.3** FPS on a single V100 GPU and yields **37.1%** AP.
Dahua Lin's avatar
Dahua Lin committed
25

WXinlong's avatar
WXinlong committed
26
## Updates
WXinlong's avatar
WXinlong committed
27
   - SOLOv2 is available. Code and trained models of SOLOv2 are released. (08/07/2020)
Xinlong Wang's avatar
Xinlong Wang committed
28
   - Light-weight models and R101-based models are available. (31/03/2020) 
WXinlong's avatar
WXinlong committed
29
   - SOLOv1 is available. Code and trained models of SOLO and Decoupled SOLO are released. (28/03/2020)
Kai Chen's avatar
Kai Chen committed
30
31


Kai Chen's avatar
Kai Chen committed
32
## Installation
WXinlong's avatar
WXinlong committed
33
This implementation is based on [mmdetection](https://github.com/open-mmlab/mmdetection)(v1.0.0). Please refer to [INSTALL.md](docs/INSTALL.md) for installation and dataset preparation.
Kai Chen's avatar
Kai Chen committed
34

WXinlong's avatar
WXinlong committed
35
36
## Models
For your convenience, we provide the following trained models on COCO (more models are coming soon).
Kai Chen's avatar
Kai Chen committed
37

WXinlong's avatar
WXinlong committed
38
39
Model | Multi-scale training | Testing time / im | AP (minival) | Link
--- |:---:|:---:|:---:|:---:
Xinlong Wang's avatar
Xinlong Wang committed
40
41
42
43
44
45
SOLO_R50_1x | No | 77ms | 32.9 | [download](https://cloudstor.aarnet.edu.au/plus/s/nTOgDldI4dvDrPs/download)
SOLO_R50_3x | Yes | 77ms |  35.8 | [download](https://cloudstor.aarnet.edu.au/plus/s/x4Fb4XQ0OmkBvaQ/download)
SOLO_R101_3x | Yes | 86ms |  37.1 | [download](https://cloudstor.aarnet.edu.au/plus/s/WxOFQzHhhKQGxDG/download)
Decoupled_SOLO_R50_1x | No | 85ms | 33.9 | [download](https://cloudstor.aarnet.edu.au/plus/s/RcQyLrZQeeS6JIy/download)
Decoupled_SOLO_R50_3x | Yes | 85ms | 36.4 | [download](https://cloudstor.aarnet.edu.au/plus/s/dXz11J672ax0Z1Q/download)
Decoupled_SOLO_R101_3x | Yes | 92ms | 37.9 | [download](https://cloudstor.aarnet.edu.au/plus/s/BRhKBimVmdFDI9o/download)
WXinlong's avatar
WXinlong committed
46
47
SOLOv2_R50_1x | No | 54ms | 34.8 | [download](https://cloudstor.aarnet.edu.au/plus/s/DvjgeaPCarKZoVL/download)
SOLOv2_R50_3x | Yes | 54ms | 37.5 | [download](https://cloudstor.aarnet.edu.au/plus/s/nkxN1FipqkbfoKX/download)
Xinlong Wang's avatar
Xinlong Wang committed
48
SOLOv2_R101_3x | Yes | 66ms | 39.1 | [download](https://cloudstor.aarnet.edu.au/plus/s/61WDqq67tbw1sdw/download)
WXinlong's avatar
WXinlong committed
49
50
SOLOv2_R101_DCN_3x | Yes | 97ms | 41.4 | [download](https://cloudstor.aarnet.edu.au/plus/s/4ePTr9mQeOpw0RZ/download)
SOLOv2_X101_DCN_3x | Yes | 169ms | 42.4 | [download](https://cloudstor.aarnet.edu.au/plus/s/KV9PevGeV8r4Tzj/download)
Xinlong Wang's avatar
Xinlong Wang committed
51
52
53
54
55

**Light-weight models:**

Model | Multi-scale training | Testing time / im | AP (minival) | Link
--- |:---:|:---:|:---:|:---:
Xinlong Wang's avatar
Xinlong Wang committed
56
57
Decoupled_SOLO_Light_R50_3x | Yes | 29ms | 33.0 | [download](https://cloudstor.aarnet.edu.au/plus/s/d0zuZgCnAjeYvod/download)
Decoupled_SOLO_Light_DCN_R50_3x | Yes | 36ms | 35.0 | [download](https://cloudstor.aarnet.edu.au/plus/s/QvWhOTmCA5pFj6E/download)
WXinlong's avatar
WXinlong committed
58
59
60
61
62
63
64
SOLOv2_Light_448_R18_3x | Yes | 19ms | 29.6 | [download](https://cloudstor.aarnet.edu.au/plus/s/HwHys05haPvNyAY/download)
SOLOv2_Light_448_R34_3x | Yes | 20ms | 32.0 | [download](https://cloudstor.aarnet.edu.au/plus/s/QLQpXg9ny7sNA6X/download)
SOLOv2_Light_448_R50_3x | Yes | 24ms | 33.7 | [download](https://cloudstor.aarnet.edu.au/plus/s/cn1jABtVJwsbb2G/download)
SOLOv2_Light_512_DCN_R50_3x | Yes | 34ms | 36.4 | [download](https://cloudstor.aarnet.edu.au/plus/s/pndBdr1kGOU2iHO/download)

*Disclaimer:*

Xinlong Wang's avatar
Xinlong Wang committed
65
- Light-weight means light-weight backbone, head and smaller input size. Please refer to the corresponding config files for details.
WXinlong's avatar
WXinlong committed
66
67
- This is a reimplementation and the numbers are slightly different from our original paper (within 0.3% in mask AP).

Kai Chen's avatar
Kai Chen committed
68

WXinlong's avatar
WXinlong committed
69
## Usage
Kai Chen's avatar
Kai Chen committed
70

Xinlong Wang's avatar
Xinlong Wang committed
71
72
73
74
### A quick demo

Once the installation is done, you can download the provided models and use [inference_demo.py](demo/inference_demo.py) to run a quick demo.

WXinlong's avatar
WXinlong committed
75
76
### Train with multiple GPUs
    ./tools/dist_train.sh ${CONFIG_FILE} ${GPU_NUM}
Kai Chen's avatar
Kai Chen committed
77

WXinlong's avatar
WXinlong committed
78
79
    Example: 
    ./tools/dist_train.sh configs/solo/solo_r50_fpn_8gpu_1x.py  8
Kai Chen's avatar
Kai Chen committed
80

WXinlong's avatar
WXinlong committed
81
82
83
84
85
### Train with single GPU
    python tools/train.py ${CONFIG_FILE}
    
    Example:
    python tools/train.py configs/solo/solo_r50_fpn_8gpu_1x.py
Kai Chen's avatar
Kai Chen committed
86

WXinlong's avatar
WXinlong committed
87
### Testing
Xinlong Wang's avatar
Xinlong Wang committed
88
89
90
91
92
93
94
    # multi-gpu testing
    ./tools/dist_test.sh ${CONFIG_FILE} ${CHECKPOINT_FILE} ${GPU_NUM}  --show --out  ${OUTPUT_FILE} --eval segm
    
    Example: 
    ./tools/dist_test.sh configs/solo/solo_r50_fpn_8gpu_1x.py SOLO_R50_1x.pth  8  --show --out results_solo.pkl --eval segm

    # single-gpu testing
WXinlong's avatar
WXinlong committed
95
96
97
    python tools/test_ins.py ${CONFIG_FILE} ${CHECKPOINT_FILE} --show --out  ${OUTPUT_FILE} --eval segm
    
    Example: 
Xinlong Wang's avatar
Xinlong Wang committed
98
99
    python tools/test_ins.py configs/solo/solo_r50_fpn_8gpu_1x.py  SOLO_R50_1x.pth --show --out  results_solo.pkl --eval segm

Kai Chen's avatar
Kai Chen committed
100

WXinlong's avatar
WXinlong committed
101
### Visualization
Kai Chen's avatar
Kai Chen committed
102

WXinlong's avatar
WXinlong committed
103
104
105
    python tools/test_ins_vis.py ${CONFIG_FILE} ${CHECKPOINT_FILE} --show --save_dir  ${SAVE_DIR}
    
    Example: 
Xinlong Wang's avatar
Xinlong Wang committed
106
    python tools/test_ins_vis.py configs/solo/solo_r50_fpn_8gpu_1x.py  SOLO_R50_1x.pth --show --save_dir  work_dirs/vis_solo
Kai Chen's avatar
Kai Chen committed
107

WXinlong's avatar
WXinlong committed
108
109
## Contributing to the project
Any pull requests or issues are welcome.
Kai Chen's avatar
Kai Chen committed
110

WXinlong's avatar
WXinlong committed
111
## Citations
Tao Kong's avatar
Tao Kong committed
112
Please consider citing our papers in your publications if the project helps your research. BibTeX reference is as follows.
Kai Chen's avatar
Kai Chen committed
113
```
Xinlong Wang's avatar
Xinlong Wang committed
114
115
116
117
118
@inproceedings{wang2020solo,
  title     =  {{SOLO}: Segmenting Objects by Locations},
  author    =  {Wang, Xinlong and Kong, Tao and Shen, Chunhua and Jiang, Yuning and Li, Lei},
  booktitle =  {Proc. Eur. Conf. Computer Vision (ECCV)},
  year      =  {2020}
Kai Chen's avatar
Kai Chen committed
119
}
Xinlong Wang's avatar
Xinlong Wang committed
120

Kai Chen's avatar
Kai Chen committed
121
```
Kai Chen's avatar
Kai Chen committed
122

WXinlong's avatar
WXinlong committed
123
124
```
@article{wang2020solov2,
Xinlong Wang's avatar
Xinlong Wang committed
125
  title={SOLOv2: Dynamic and Fast Instance Segmentation},
WXinlong's avatar
WXinlong committed
126
  author={Wang, Xinlong and Zhang, Rufeng and  Kong, Tao and Li, Lei and Shen, Chunhua},
Xinlong Wang's avatar
Xinlong Wang committed
127
  journal={Proc. Advances in Neural Information Processing Systems (NeurIPS)},
WXinlong's avatar
WXinlong committed
128
129
130
  year={2020}
}
```
WXinlong's avatar
WXinlong committed
131
132
133
134

## License

For academic use, this project is licensed under the 2-clause BSD License - see the LICENSE file for details. For commercial use, please contact [Xinlong Wang](https://www.xloong.wang/) and  [Chunhua Shen](https://cs.adelaide.edu.au/~chhshen/).