GitLab

[Feature] Add fast_conv_bn_eval option in ConvModule for fast validation and training in Eval mode (#2807)

* support rfsearch

* add labs for rfsearch

* format

* format

* add docstring and type hints

* clean code
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* rm unused func

* update code

* update code

* update code

* update  details

* fix details

* support asymmetric kernel

* support asymmetric kernel

* Apply suggestions from code review
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Apply suggestions from code review

* Apply suggestions from code review
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Apply suggestions from code review

* add unit tests for rfsearch

* set device for Conv2dRFSearchOp

* Apply suggestions from code review
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* remove unused function search_estimate_only

* move unit tests

* Update tests/test_cnn/test_rfsearch/test_operator.py
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Apply suggestions from code review
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmcv/cnn/rfsearch/operator.py
Co-authored-by: Yue Zhou <592267829@qq.com>

* change logger

* Update mmcv/cnn/rfsearch/operator.py
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
Co-authored-by: lzyhha <819814373@qq.com>
Co-authored-by: Zhongyu Li <44114862+lzyhha@users.noreply.github.com>
Co-authored-by: Yue Zhou <592267829@qq.com>

[Fix] Fix skip_layer for RF-Next (#2489)

* judge skip_layer by fullname

* lint

* skip_layer first

* update unit test

* add layer scale

* add layer scale

* add layer scale

* Update mmcv/cnn/bricks/transformer.py
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmcv/cnn/bricks/transformer.py
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* add layer scale

* move LayerScale

* add layer_scale_init_value

* add typehint

* fix for tensor with any dim

* fix layer scale rule

* fix layer scale rule

* fix test

* add docs
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Remove runner, parallel, engine and device

* fix format

* remove outdated docs

* migrate many functions to mmengine

* remove sync_bn.py

* change MODELS to mmengine, delete basemodule

* fix unit test

* remove build from cfg

* add comment and rename TARGET_MODELS to registry

* refine cnn docs

* remove unnecessary check

* refine as comment

* refine build_xxx_conv error message

* fix lint

* fix import registry from mmcv

* remove unused file

* add pyupgrade

* add options for pyupgrade

* minor refinement

* first commit

* Add copyright pre-commit-hook

* add ops (spconv) of mmdet3d

* fix typo

* refactor code

* resolve comments in #1452

* fix compile error

* fix bugs

* fix bug

* transform from 'types.h' to 'extension.h'

* fix bug

* transform from 'types.h' to 'extension.h' in parrots

* add extension.h in pybind.cpp

* add unittest

* Recover code

* (1) Remove prettyprint.h
(2) Switch `T` to `scalar_t`
(3) Remove useless lines
(4) Refine example in docstring of sparse_modules.py

* (1) rename from `cu.h` to `cuh`
(2) remove useless files
(3) move cpu files to `pytorch/cpu`

* reorganize files

* Add docstring for sparse_functional.py

* use dispatcher

* remove template

* use dispatch in cuda ops

* resolve Segmentation fault

* remove useless files

* fix lint

* fix lint

* fix lint

* fix unittest in test_build_layers.py

* add tensorview into include_dirs when compiling

* recover all deleted files

* fix lint and comments

* recover setup.py

* replace tv::GPU as tv::TorchGPU & support device guard

* fix lint
Co-authored-by: hdc <hudingchang.vendor@sensetime.com>
Co-authored-by: grimoire <yaoqian@sensetime.com>

* [Enhance] Use PyTorch HSwish implementation.

* fix conv test

* upgrade version

* add version comments

* fix BaseTransformerLayer

* Add BaseTransformerLayer unit test without ffn_cfg embed_dims

* Optimize code logic

* [Fix] align hsigmoid with pytorch official

* [Fix] add warnings for Hsigmoid

* [Fix] fix format

* [Fix] add unittest

* [Fix] fix docstring

* [Fix] fix some warnings in unittest

* [Impl] standardize some warnings

* [Fix] fix warning type in test_deprecation

* [Fix] fix warning type

* [Fix] continue fixing

* [Fix] fix some details

* [Fix] fix docstring

* [Fix] del useless statement

* [Fix] keep compatibility for torch < 1.5.0

* [Feature] Add revert_sync_batchnorm

* support mmsyncbn (to be tested)

* Test passed

* Update docstring, rename the test file

* remove test_sync_bn

* add comment

* add mmcv.ops check

* Add a comment

* Add notes and relax test req
Co-authored-by: gaotongxiao <gaotongxiao@gmail.con>

* restrict the warning message

* and an important keyword in warning description

* a more elegant way of condition

* link format code too long

* fix the stupid spelling mistake

* Use issubclass to restrict warning message.

* maybe this version is more elegant.

* conv + bias + norm warning pytest

* 'created' a warning, hahaha

* isort and yapf format revision

* isort and yapf format revision

* flake8 fail issue

* I have to right this way in order to solve the conflicts between yapf and flake8, sigh...

* fixed test bug

* Add ruby windows installer source.

* Simplified the code and remove ruby source from CONTRIBUTING.md

* use _BatchNorm to simplify the code

* bug fix and add instanceNorm case into warning

* change the warning message to make it more clear

* fix unit test

* [Docs] Add header for files

* change to OpenMMLab

* add headers for .cpp, .cu, .h, .cuh

* replace Open-MMLab with OpenMMLab

* empty tensor inference backward continity

* update

* add 3d

* minor changes

* change to modulist

* change to Sequential

* replace dropout with attn_drop and proj_drop in MultiheadAttention

* add operation_name for attn

* add drop path and move all ffn args to ffncfgs

* fix typo

* fix a bug when use default value of ffn_cfgs

* fix ffns

* add deprecate warning

* fix deprecate warning

* change to pop kwargs

* support register FFN of transformer

* support batch first

* fix batch first wapper

* fix forward wapper

* fix typo

* fix lint

* add unitest for transformer

* fix unitest

* fix equal

* use allclose

* fix comments

* fix comments

* change configdict to dict

* move drop to a file

* add comments for drop path

* add noqa 501

* move bnc wapper to MultiheadAttention

* move bnc wapper to MultiheadAttention

* use dep warning

* resolve comments

* add unitest:

* rename residual to identity

* revert runner

* msda residual to identity

* rename inp_identity to identity

* fix name

* fix transformer

* remove key in msda

* remove assert for key
Co-authored-by: HIT-cwh <2892770585@qq.com>
Co-authored-by: bkhuang <congee524@gmail.com>
Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com>

* [Fix] Support names of base classes matching in init_cfg

* revise bool to len

* [Feature] Add truncated normal weight init

* [Feature] Add truncated normal weight init

* [Feature] Add truncated normal weight init

* update docstring

* delete modelA.pth

* modify according to comment

* use kstest to check truncated normal

* delete modelA.pth

* fix test.txt

* fix generalized attention fp16

* fix building without gpu error

* add comment

* Cast tensor at initialization

* [Fix] Config deep copy in initialize_override

* add asserts&comments

* add test

* test org init_cfg

* test override without name

* typo

* add model registry

* fixed infer scoep

* fixed build func

* add docstring

* add md

* support multi level

* clean comments

* add docs

* fixed parent

* add more doc

* add value error, add docstring

* fixed docs

* change to local/global search

* resolve comments

* fixed test

* update some docstring

* update docs (minior)

* update docs

* update docs

* [Refactoring]Approve init_cfg

* revise unit test

* revise according to comments

* add comment for wholemodule

* [Refactoring] Add Caffe2Xavier Initializer

* fix lint

* fix bug of convmodule

* fix bug of convmodule

* fix unitest

* remove assert

* [Feature] support to calculate FLOPs of GN, IN, LN

* Update flops_counter.py

* Update flops_counter.py

* add initializers and BaseModule for unified parameter initialization

* fix circle import

* bug fix

* add is_init flag in BaseModule

* fix docstring

* sort import and fix doc format

* fix bug

* fix docformat and double quote string

* fix import sort

* import sort

* sort import

* revise according to comments

* fix doc format

* revise according to comments

* revise import and fix typo

* polish code

* revise minors

* revice minors

* revise apply function

* revise bias initialization with probability

* add type test for bias_prob

* revise minors

* avoid recursion error when using customized norm layer without _abbr_

* fix unittest bug

* add clamp without unittest

* add clamp-act with unit test

* fix name bug

* use logical and

* fix logical_and

* fix linting

* rename ClampLayer to Clamp

* rename ClampLayer to Clamp
Co-authored-by: nbei <631557085@qq.com>

* enhance hsigmoid

* delete int