GitLab

* Fix for roi_align export

* Disable interpolate script module tests

Disable test until export of interpolate script module to ONNX is fixed

* add checkout/assert in roi_pool

* add checkout/assert in roi_align

* move check_roi_boxes_shape func to ops/_utils.py

* add tests

* fix CI

* fix CI
Co-authored-by: Guanheng Zhang <zhangguanheng@devfair0197.h2.fair>

* fix googlenet no aux logits

* small fix
Co-authored-by: eellison <eellison@fb.com>

* Aligned flag in the interfaces

* Aligned flag in the impl, and remove unused comments

* Handling empty bin in forward

* Remove raise error in roi_width

* Aligned flag in the Testcodes

* ci: Add verbosity to pytest results, store in ci

Makes the pytest runs for building conda packages more verbose and
stores the results for viewing inside of CircleCI
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

* test: Skip inception v3 in test_quantized_models

Was causing timeouts on circleci due to long run time, re-enable when
tests can be brought to a reasonable time again.
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

* feat: Added __repr__ attribute to GeneralizedRCNNTransform

Added more details to default __repr__ attribute for printing.

* fix: Put back relative imports

* style: Fixed pep8 compliance

Switched strings with  syntax to f-strings.

* test: Added test for GeneralizedRCNNTransform __repr__

Checked integrity of __repr__ attribute

* test: Fixed unittest for __repr__

Fixed the formatted strings in the __repr__ integrity check for GeneralizedRCNNTransform

* fix: Fixed f-strings for earlier python versions

Switched back f-strings to .format syntax for Python3.5 compatibility.

* fix: Fixed multi-line string

Fixed multiple-line string syntax for compatibility

* fix: Fixed GeneralizedRCNNTransform unittest

Fixed formatting of min_size argument of the resizing part

* Force object annotiation to be an array

* Remove unecessary parentheses

* Change object check

* Remove check for list

* Add test coverage to xml parsing

* Tidy up whitespace

* Fix indentation

* Force object annotiation to be an array

* Remove unecessary parentheses

* Change object check

* Remove check for list

* Add test coverage to xml parsing

* Tidy up whitespace

* torchscriptable functions for video io (#1653)

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1653



created new torchscriptable video io functions as part of the api: read_video_meta_data_from_memory and read_video_from_memory.

Updated the implementation of some of the internal functions to be torchscriptable.

Reviewed By: stephenyan1231

Differential Revision: D18720474

fbshipit-source-id: 4ee646b66afecd2dc338a71fd8f249f25a3263bc

* BugFix
Co-authored-by: Jon Guerin <54725679+jguerin-fb@users.noreply.github.com>

* adding new_empty_tensor symbolic

* flake8

* fix for feedback

* skipping the ORT test

* fix for ORT test

* initial fix

* outsourced num bands lookup

* fix doc

* added pillow version requirement

* simplify number of bands extraction

* remove unrelated change

* remove indirect dependency on pillow>=5.2.0

* extend docstring to transform

* bug fix

* added test

* Fix AnchorGenerator if moving from one device to another

* Fixes for the test

* Testing CI

* Disable tests for Pillow 7

* scriptability checks

* tests upds

* linter upds

* linter upds

* upds

* tuple list changes

* linter updates

* remove windows skips from video_utils tests, now that they pass

* replace lambda in videoclips in order to be pickled on windows and update tests

* out_place checks

* lint ups

* tgz updates

* tgz updates

* tgz updates

* add scriptable transform: center_crop

* add test: center_crop

* add scriptable transform: five_crop

* add scriptable transform: five_crop

* add scriptable transform: fix minor issues

* Add Deformable Convolution operation.

This adds the deformable convolution operation, as described in Deformable Convolutional Networks (https://arxiv.org/abs/1703.06211).

- The code is based on https://github.com/open-mmlab/mmdetection/blob/master/mmdet/ops/dcn/src/deform_conv_cuda.cpp ; the whole code was modified and refactored to remove redundancies and increase clarity, and to adapt it to torchvision.

- The CPU part is a direct copy of the CUDA code; it might make sense to do follow-up adjustments in the CPU code to simplify it / optimize it, or to reuse functionality between CPU and CUDA..

- We also add tests (with a non-trivial set of parameters); they can be made more robust by randomizing the parameters and executing multiple times.

* Update DeformConv to be more consistent w/ Conv2d

* rename some variables and arguments to match Conv2d;
* add optional bias;
* add weight, offset and bias as module parameters;
* remove the n_parallel_imgs parameter;
* Fix __repr__;
* etc..

Initialization of weight and bias is the same as in Conv2d, and
initialization of offsets to zero is the same as in the paper.

This also includes some other small unrelated fixes/improvements.

* Apply clang-format in DeformConv files.

* Import Optional type annotation

* Remove offset param from DeformConv2d module

- We pass the offset in the forward of DeformConv2d, instead of having
an internal parameter. This adds some complexity to creating the module
(e.g. now you have to worry about the output size, to create the
offset), but it gives more flexibility.
- We also use make_tuple for tuple creation, in an attempt to fix error
w/ older compilers.

* Replace abs by std::abs

Old gcc versions were giving wrong results here, because they would
resolve abs as int -> int, thus causing undesired truncation. Replacing
abs by std::abs should allow for correct overloading of abs as float -> float.

* Reorder declarations for clarity

* Reorder weight and offset args in deform_conv2d

We place offset arg before the weight arg, to be more
consistent with DeformConv2d.forward(input, offset)

* Replace abs by std::abs in DeformConv_cuda

* Add tests for results in script vs eager mode

This copies some logic from `test_jit.py` to check that a TorchScript'ed
model's outputs are the same as outputs from the model in eager mode.

To support differences in TorchScript / eager mode outputs, an
`unwrapper` function can be provided per-model.

* Fix inception, use PYTORCH_TEST_WITH_SLOW

* Update

* Remove assertNestedTensorObjectsEqual

* Add PYTORCH_TEST_WITH_SLOW to CircleCI config

* Add MaskRCNN unwrapper

* fix prec args

* Remove CI changes

* update

* Update

* remove expect changes

* Fix tolerance bug

* Fix breakages

* Fix quantized resnet

* Fix merge errors and simplify code

* DeepLabV3 has been fixed

* Temporarily disable jit compilation

* almost working...

* respond to comments

* add empty tensor op, handle different output types in generalized rcnn

* clean ups

* address comments

* more changes

* it's working!

* torchscript bugs

* add script/ eager test

* eval script model

* fix flake

* division import

* py2 compat

* update test, fix arange bug

* import division statement

* fix linter

* fixes

* changes needed for JIT master

* cleanups

* remove imagelist_to

* requested changes

* Make FPN backwards-compatible and torchscript compatible

We remove support for feature channels=0, but support for it was already a bit limited

* Fix ONNX regression

* code changes to enable onnx export for keypoint rcnn

* add import

* fix copy paste error

* disable test

* disable profiling

* Update test_onnx.py

* enable faster rcnn test

* flake8

* smaller image size

* set min/max

* Simlify and organize test_ops.

We perform the following:

- Simplify the functions slow_roi_pooling, slow_ps_roi_pooling, slow_ps_roi_align and bilinear_interpolate (including finding and removing a semi-bug in slow_ps_roi_pooling, which used bin_w instead of bin_h);
- Wrote a slow_roi_align function, that was missing;
- Create a base class testing all combinations of forward/backward, cpu/cuda, contiguous/non-contiguous;
- Organize all testing inside the base class with _test_forward and _test_backward (which can be easily overriden if a parciular op needs something different); an Op class then only needs to implement fn, get_script_fn, and expected_fn.

A few points:
- We are using the same inputs for all tests, and not trying all possible inputs in the domain of a given operation. One improvement would be to test more diverse inputs, and to personalize the inputs for some ops (e.g. different inputs for pooling ops and align ops).
- Running all tests is quite slow (~1 min only for CPU tests), so that can possibly be improved.

* Reduce input size used in gradcheck.

gradcheck can be quite costly, and it was causing OOM errors and making
the tests slow. By reducing the size of the input, the test speed is
down to 3 seconds for the CPU tests.

Other points:
- We remove an unused namedtuple;
- We inherit from object for better Python 2 compatibility;
- We remove a hardcoded pool_size from the TorchScript functions, and
add it as a parameter instead.

* Replace Tensor by torch.Tensor in type annotations.

This should fix lint errors.

* Fix inconsistent NMS implementation

* Improve tests for NMS

* Remove unnecessary using statement

* Add Scriptable Transform: Grayscale

* add scriptable transforms: rgb_to_grayscale

* add scriptable transform: rgb_to_grayscale

* add scriptable transform: rgb_to_grayscale

* add scriptable transform: rgb_to_grayscale

* update code: rgb_to_grayscale

* add test: rgb_to_grayscale

* update parameters: rgb_to_grayscale

* add scriptable transform: rgb_to_grayscale

* update rgb_to_grayscale

* update rgb_to_grayscale

* Disable C++ models from being compiled without explicitly being asked for

* Fix import in tests, which are already disabled

* Unify video metadata in VideoClips

* Bugfix

* Make tests a bit more robust

* Improve readability of affine transformation code

* Make shear transformation area preserving

The previous shear implementation did not preserve area, and we
implement a version that does.

The formula used was verified with the following sympy code:

from sympy import Matrix, cos, sin, tan, simplify
from sympy.abc import x, y, phi

Xs = Matrix(
        [[1, -tan(x)],
         [0, 1]]
        )

Ys = Matrix(
        [[1, 0],
         [-tan(y), 1]]
        )

R = Matrix(
        [[cos(phi), -sin(phi)],
         [sin(phi), cos(phi)]]
        )

RSS = Matrix(
        [[cos(phi - y)/cos(y), -cos(phi - y)*tan(x)/cos(y) - sin(phi)],
         [sin(phi - y)/cos(y), -sin(phi - y)*tan(x)/cos(y) + cos(phi)]])

print(simplify(R * Ys * Xs - RSS))

One thing that is not clear (and could be tested) is whether avoiding
the explicit products and calculations in _get_inverse_affine_matrix
really gives performance benefits - compared to doing the explicit
calculation done in _test_transformation.

* Use np.matmul instead of @

The @ syntax is not supported in Python 2.

* Support Exporting Mask Rcnn to ONNX

* update tetst

* add control flow test

* fix

* update test and fix img_shape

* add quantized models

* Modify mobilenet.py documentation and clean up comments
Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Move fuse_model method to QuantizableInvertedResidual and clean up args documentation
Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Restore relu settings to default in resnet.py
Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Fix missing return in forward
Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Fix missing return in forwards
Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Change pretrained -> pretrained_float_models
Replace InvertedResidual with block

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Update tests to follow similar structure to test_models.py, allowing for modular testing

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Replace forward method with simple function assignment

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Fix error in arguments for resnet18

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* pretrained_float_model argument missing for mobilenet

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* reference script for quantization aware training and post training quantization

* reference script for quantization aware training and post training quantization

* set pretrained_float_model as False and explicitly provide float model

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Address review comments:
1. Replace forward with _forward
2. Use pretrained models in reference train/eval script
3. Modify test to skip if fbgemm is not supported

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Fix lint errors.
Use _forward for common code between float and quantized models
Clean up linting for reference train scripts
Test over all quantizable models

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Update default values for args in quantization/train.py

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Update models to conform to new API with quantize argument
Remove apex in training script, add post training quant as an option
Add support for separate calibration data set.

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Fix minor errors in train_quantization.py

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Remove duplicate file

* Bugfix

* Minor improvements on the models

* Expose print_freq to evaluate

* Minor improvements on train_quantization.py

* Ensure that quantized models are created and run on the specified backends
Fix errors in test only mode

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Add model urls

* Fix errors in quantized model tests.
Speedup creation of random quantized model by removing histogram observers

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Move setting qengine prior to convert.

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Fix lint error

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Add readme.md

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Readme.md

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Fix lint

* Add adjustment operations for RGB Tensor Images.

Right now, we have operations on PIL images, but we want to have a version of the opeartions that act directly on Tensor images.

Here, we add such operations for adjust_brightness, adjust_contrast and adjust_saturation.

In PIL, those functions are implemented by generating an degenerate image from the first, and then interpolating them together.
- https://github.com/python-pillow/Pillow/blob/master/src/PIL/ImageEnhance.py
- https://github.com/python-pillow/Pillow/blob/master/src/libImaging/Blend.c

A few caveats:
* Since PIL operates on uint8, and the tensor operations might be on float, we can get slightly different values because of int truncation.
* We assume here the images are RGB; in particular, to handle an alpha channel, we need to check whether it is present, in which case we copy it to the final image.

* Keep dtype and use broadcast in adjust operations

- We make our operations have input.dtype == output.dtype, at the cost of
adding a few type checks and branches.

- By using Tensor broadcast, we can simplify the calls to _blend.

* Use is_floating_point to check dtype.

* Remove unpacking in tuple

It seems Python 2 does not support this type of unpacking, so it broke
Python 2 builds. This should fix it.

* Add from __future__ import division for Python 2