GitLab

[Feature]: Add Rotated ROI align op for pytorch (cpu&cuda), parrots (cpu&cuda) and onnxruntime (cpu) (#933)

* add roi_align_rotated

* code format

* Add align key to roi align rotated

* Add clockwise for rotated roi align

* fix bugs in onnx export

* Add docstring for RoIAlignRotated

* remove cuda unittest

* Reformat c++ code

* add onnx roi align rotated file

* fix unittest

* Add  cpu and float64 of cuda support for parrots

* code format

* Add unified header to roi align rotated
Co-authored-by: luopeichao <luopeichao@sensetime.com>

* [Bug] Fix DeformConv2d bias error and add tests

* fix repr

* revise tests

* lint

* replace floor/ceil to floorf/ceilf

to eliminate cuda compilation errors in the latest version of Visual Studio 16.9

* fix lint error

* fix lint

* fix lint

* Update .pre-commit-config.yaml

* Update .pre-commit-config.yaml
Co-authored-by: wangruohui <12756472+wangruohui@users.noreply.github.com>

* [Docs] Docstring for DeformConv

* fix docstring

* fix according to comments

* revise according to comments

* lint

* add onnxruntime custom op grid_sample

* update code

* update code

* update code

* update code

* update code

* update code

* update code

* update code

* update code

* update code

* update code

* update code

* update code

* update code

* add grid sample trt support

* fix align_corners=True

* change 'intep' to 'interp', fix bugs of grid sampler, better test script

* remove unused import

* move source comment to the top of trt_grid_sampler_kernel.cu

* add upfirdn2d op

* fix bug in pybind

* add fused bias leakyrelu

* fix bug in fused-bias-leakyrelu

* fix lint error

* fix bug in build cpu version

* fix bug in build cpu version

* fix lint

* fix comment from zww
Co-authored-by: zhangshilong <zhangshilong@sensetime.com>

* add dcn tensorrt plugin

* prepare for fp16 support

* fix for lint

* limit column buffer

* add docstring to memcpyPermute

* delete all parrots file
add bbox_overlaps new parrots op impl

* support first new impl parrts op (bbox_overlaps)(success test)

* add box_iou_rotated op, test succeed

* add carafe and carafe_naive op, test succeed (one parrots bug need fix)

* add cc_attention op, test success

* add corner_pool op, test success

* add parrots op deform_conv, test success

* add deform_roi_pool op, test success (but has question)

* add focal loss op, test success (gradcheck)

* add masked_conv2d op, test success

* add modulated_deform_conv op, test success

* add nms and nms_rotated op, test success

* add psamask op, test success

* add roi_align op, test_success

* add roi_pool op, test success

* add sync_bn op, test success

* add tin_shift op, test success

* fix test_deform_roi_pool, add parrots test

* skip test_onnx because parrots does not support onnx

* fix c++ lint

* fix python lint

* fix python lint

* start trt plugin prototype

* Add test module, modify roialign convertor

* finish roi_align trt plugin

* fix conflict of RoiAlign and MMCVRoiAlign

* fix for lint

* fix test tensorrt module

* test_tensorrt move import to test func

* add except error type

* add tensorrt to setup.cfg

* code format with yapf

* fix for clang-format

* move tensorrt_utils to mmcv/tensorrt, add comments, better test module

* fix line endings, docformatter

* isort init, remove trailing whitespace

* add except type

* fix setup.py

* put import extension inside trt setup

* change c++ guard, update pytest script, better setup, etc

* sort import with isort

* sort import with isort

* move init of plugin lib to init_plugins.py

* add scatternd, nms plugin (WIP)

* fix bugs of trt_nms

* add trt nms test module

* fix bugs of scatternd

* code optimize, add comment about nms kernel

* fix transform_if bug of trt_nms_kernel

* fix struct name

* default nms offset=0, fix bugs of batched input

* format with clang-format

* onnx preprocess

* much better nms implementation, no need to transfer memory between host and device

* update preprocess_onnx

* parse constant tensor from initializer in preprocess_onnx

* update nms

* remove unnecessary codes

* workspace aligned address

* format trt_plugin_helper.hpp

* fix index memory bugs

* set alignment to 16 by default

* fix lint

* fix nms offset

* fix bugs of preprocess onnx

* update test for nms

* tensorrt only accept int32, not int64

* update nms comments

* fix indexing for scores in nms

* update trt temp

* make trt-nms compatiable to #803

* fix lint

* add docstring to trt_nms_kernel.cuda, add description to preprocess_onnx

* add comment to score indexing

* fix bugs of max output boxes
Co-authored-by: maningsheng <maningsheng@sensetime.com>

* fix bug of DCN

* fix clang=format

* add header to std::iota

* fix lint

* fix lint for cpp

* add ort nms op

* fit lint check

* fix lint check

* update code

* fix lint check

* update code

* update code

* update code

* update code

* update code

* update code

* add scatter plugin

* fix bugs of scatternd

* add trt scatternd plugin

* format code with clang-format

* add test for scatternd

* skip test_tensorrt in CI

* remove unused variable
Co-authored-by: maningsheng <maningsheng@sensetime.com>

* fix mmcv ci for parrots

* fix mmcv ci

* fix lint

* start trt plugin prototype

* Add test module, modify roialign convertor

* finish roi_align trt plugin

* fix conflict of RoiAlign and MMCVRoiAlign

* fix for lint

* fix test tensorrt module

* test_tensorrt move import to test func

* add except error type

* add tensorrt to setup.cfg

* code format with yapf

* fix for clang-format

* move tensorrt_utils to mmcv/tensorrt, add comments, better test module

* fix line endings, docformatter

* isort init, remove trailing whitespace

* add except type

* fix setup.py

* put import extension inside trt setup

* change c++ guard, update pytest script, better setup, etc

* sort import with isort

* sort import with isort

* move init of plugin lib to init_plugins.py

* resolve format and add test dependency: tensorrt

* tensorrt should be installed from source not from pypi

* update naming style and input check

* resolve lint error
Co-authored-by: maningsheng <maningsheng@sensetime.com>

* add ort custom op roialign

* Update roi_align.py

* Update test_onnx.py

* Update test_onnx.py

* Update test_onnx.py

* Update test_onnx.py

* Update onnxruntime_register.cpp

* Update roiAlign.h

* Update roiAlign.cpp

* lint modification

* update roiAlign.cpp

* lint check

* lint check

* lint fix

* lint fix

* fix lint

* add link to commit
Co-authored-by: maningsheng <maningsheng@sensetime.com>

* add iof mode for box_iou_rotated

* update doc

* fix lint

* fix lint

* fix lint

* fix lint

* fix lint

* add onnx support to roi_align and roi_pool

* add softnms ort support

* fix for lint

* format cpp code with clang-format:google

* add new empty line to the end of head files in onnxruntime

* update to pytorch1.7

* add test of softnms to onnxruntime

* fix for lint

* remote print in ops/info.py

* change import order, fix for flake8

* fix include

* add assert torch>=1.7.0

* [doc]: add document for onnxruntime custom operator

* update onnxruntime version to v1.5.1 for softnms

* remove doc menu

* Resolve lint for markdown

* resolve naming style in onnxruntime_op.md

* Use old cpp apis, optimize test_onnx.py

* Fixing strings in tests/test_ops/test_onnx.py

* code format with yapf

* fix soft_nms parrot

* add import in onnxruntime setup, avoid conflict

* fix doc and add assert

* change cpp guard
Co-authored-by: maningsheng <maningsheng@sensetime.com>

* fix grad_output uncontiguous

* add comments for uncontiguous

* add const to nms_rorate in pybind

* fix test nms rotated

* skip test instead of passing it

* fix lint

* update pytest skip syntax

* support aligned and parrots cpu for  box_iou_roatetd

* add aligned doc

* fix lint

* fix lint

* fix lint

* fix lint

* fix bug

* fix bug

* fix bug

* fix lint

* fix lint

* fix bug

* fix bug

* fix roi_align ci for parrots

* fix lint

* add box_iou_rotated, ml_nms_rotated and nms_rotated

* fix lint

* fix lint

* fix .py lint

* fix cpp lint

* add newline at the end

* add new line

* fix unittest

* config google style

* fix lint

* lint

* lint

* yapf

* update

* fix lint

* fix lint

* fix lint

* fix

* fix format

* fix format

* add modified from

* add docstring and update others

* update docstring

* update docstring

* update

* fix bug

* fix bug

* fix bug
Co-authored-by: Cao Yuhang <yhcao6@gmail.com>

* [feature]: Support convertion from Pytorch to ONNX for detection models

* [fix] RuntimeError with integer divide when converting to onnx

* Resolve pylint error

* add comment

* add onnx support to roi_align and roi_pool

* add test on roi_align and roi_pool

* fix conflict with master

* fix issues when converting deformable convolution to onnx

* keep  and  for interface consistency
Co-authored-by: maningsheng <maningsheng@sensetime.com>

* add unittest for onnx convert

* build onnx and onnxruntime in CI

* skip onnx op unit test while using CUDA

* fix offset==0 case in NMS

* remove tmp file used in test

* delete tmp file before assert so that we can remove the tmp file anyway

* Support to specify LR of DCN's conv_offset

* Resolve comments & add unit test

* Resolve formats

* Fix CI for DCN

* Mock DCN when cpu only

* Use mock for cpu testing

* Fix docstring and support ModulatedDCN

* set offset_lr_mult as dcn's arguments, link CU-49u01p

* fix lr bug

* fall back to set LR in constructor

* resolve comments

* add torchvision roi_align with aligned=True

* fix for lint test

* fix for lint test2

* format use yapf

* Mv wrappers into bricks and use wrappers in registry

* resolve import issues

* fix import issues

* set nn op forward to torch 1.6.1

* fix CI bug and add warning

* Fix CI by using patch mock

* mv warnings inside deprecated module's initialization

* Add windows instruction and fix compilation bug

* reformat codebase

* Fix function types

* Remove const

* Support to split batched_nms when box number is too large

* mv data from gpu to cpu

* Set split_thr through nms_cfg

* clean code

* Update motivation in docstring

* fix typos