GitLab

* Make copy of targets in GeneralizedRCNNTransform

* Fix flake8

* Adds as_tensor to functional.py

Similar functionality to to_tensor without the default conversion to float and division by 255.
Also adds support for Image mode 'L'.

* Adds tests to AsTensor()

Adds tests to AsTensor and removes the conversion to float and division by 255.

* Adds AsTensor to transforms.py

Calls the as_tensor function in functionals and adds the function AsTensor as callable from transforms.

* Removes the pic.mode == 'L'

This was handled by the else condition previously so I'll remove it.

* Fix Lint issue

Adds two line breaks between functions to fix lint issue

* Replace from_numpy with as_tensor

Removes the extra if conditionals and replaces from_numpy with as_tensor.

* Renames as_tensor to pil_to_tensor

Renames the function as_tensor to pil_to_tensor and narrows the scope of the function.  At the same time also creates a flag that defaults to True for swapping to the channels first format.

* Renames AsTensor to PILToImage

Renames the function AsTensor to PILToImage and modifies the description.  Adds the swap_to_channelsfirst boolean variable to indicate if the user wishes to change the shape of the input.

* Add the __init__ function to PILToTensor 

Add the __init__ function to PILToTensor since it contains the swap_to_channelsfirst parameter now.

* fix lint issue

remove trailing white space

* Fix the tests

Reflects the name change to PILToTensor and the parameter to the function as well as the new narrowed scope that the function only accepts PIL images.

* fix tests

Instead of undoing the transpose just create a new tensor and test that one.

* Add the view back

Add img.view(pic.size[1], pic.size[0], len(pic.getbands())) back to outside the if condition.

* fix test

fix conversion from torch tensor to PIL back to torch tensor.

* fix lint issues

* fix lint

remove trailing white space

* Fixed the channel swapping tensor test

Torch tranpose operates differently than numpy transpose.  Changed operation to permute.

* Add mode='F'

Add mode information when converting to PIL Image from Float Tensor.

* Added inline comments to follow shape changes

* ToPILImage converts FloatTensors to uint8

* Remove testing not swapping

* Removes the swap_channelsfirst parameter

Makes the channel swapping the default behavior.

* Remove the swap_channelsfirst argument

Remove the swap_channelsfirst argument and makes the swapping the default functionality.

* freeze layers only if pretrained backbone is used

If pretrained backbone is not used and one intends to train the entire network from scratch, no layers should be frozen.

* function argument to control the trainable features

Depending on the size of dataset one might want to control the number of tunable parameters in the backbone, and this parameter in hyper parameter optimization for the dataset. It would be nice to have this function support this.

* ensuring tunable layer argument is valid

* backbone freezing in fasterrcnn_resnet50_fpn

Handle backbone freezing in fasterrcnn_resnet50_fpn function rather than the resnet_fpn_backbone function that it uses to get the backbone.

* remove layer freezing code

layer freezing code has been moved to fasterrcnn_resnet50_fpn function that consumes resnet_fpn_backbone function.

* correcting linting errors

* correcting linting errors

* move freezing logic to resnet_fpn_backbone

Moved layer freezing logic to resnet_fpn_backbone with an additional parameter.

* remove layer freezing from fasterrcnn_resnet50_fpn

Layer freezing logic has been moved to resnet_fpn_backbone. This function only ensures that the all layers are made trainable if pretrained models are not used.

* update example resnet_fpn_backbone docs

* correct typo in var name

* correct indentation

* adding test case for layer freezing in faster rcnn

This PR adds functionality to specify the number of trainable layers while initializing the faster rcnn using fasterrcnn_resnet50_fpn function. This commits adds a test case to test this functionality.

* updating layer freezing condition for clarity

More information in PR

* remove linting errors

* removing linting errors

* removing linting errors

* Check target boxes input on generalized_rcnn.py

* Fix target box validation in generalized_rcnn.py

* Add tests for input validation of detection models

* feat: Added eps argument to FrozenBatchNorm2d

* test: Added unittest for eps addition in FrozenBatchNorm2d

See #2169

* fix: Reverted forward changes for JIT fuser

* fix: Added back n argument for backward-compatibility

* fix: Fixed FrozenBatchNorm2d forward

Added back eps

* feat: Specified deprecation warnings in FrozenBatchNorm2d

* test: Added unittest for deprecation warninig in FrozenBatchNorm2d

* style: Fixed lint

* style: Fixed block comment lint

* feat: Restored support of tuple of Tensors for roi_align & roi_pool

* test: Added unittest for Tensor sequence support by region pooling

* test: Fixed typo in unittest

* test: Fixed data type

* test: Fixed roi pooling tensor unittest

* test: Fixed box format conversion

* Fixing nms on boxes when no detection

* test

* Fix for scale_factor computation

* remove newline

* Fix for mask_rcnn dynanmic axes

* Clean up

* Update transform.py

* Fix for torchscript

* Fix scripting errors

* Fix annotation

* Fix lint

* Fix annotation

* Fix for interpolate scripting

* Fix for scripting

* refactoring

* refactor the code

* Fix annotation

* Fixed annotations

* Added test for resize

* lint

* format

* bump ORT

* ort-nightly version

* Going to ort 1.1.0

* remove version

* install typing-extension

* Export model for images with no detection

* Upgrade ort nightly

* update ORT

* Update test_onnx.py

* updated tests

* Updated tests

* merge

* Update transforms.py

* Update cityscapes.py

* Update celeba.py

* Update caltech.py

* Update pkg_helpers.bash

* Clean up

* Clean up for dynamic split

* Remove extra casts

* flake8

* Fix missing compilation files for video-reader

* Disable IO tests in travis

* feat: Added number of features in FrozenBatchNorm2d repr

While BatchNorm layers have extensive information in their repr, FrozenBatchNorm2d has one

* refactor: Refactored FrozenBatchNorm2d __repr__

* test: Added unittest for FrozenBatchNorm2d __repr__

* style: Removed blank lines in test_ops

* refactor: Avoids creating an extra attribute for __repr__

* style: Switched __repr__ to f-string

Since support of Python version ealier than 3.6 have been dropped, f-string can be used.

* fix: Fixed typo in __repr__

* style: Switched unittest .format to f-string

* only display extensions if available

* add tests

* fix lint

* Fixing nms on boxes when no detection

* test

* Fix for scale_factor computation

* remove newline

* Fix for mask_rcnn dynanmic axes

* Clean up

* Update transform.py

* Fix for torchscript

* Fix scripting errors

* Fix annotation

* Fix lint

* Fix annotation

* Fix for interpolate scripting

* Fix for scripting

* refactoring

* refactor the code

* Fix annotation

* Fixed annotations

* Added test for resize

* lint

* format

* bump ORT

* ort-nightly version

* Going to ort 1.1.0

* remove version

* install typing-extension

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/2077

Pull Request resolved: https://github.com/facebookresearch/SlowFast/pull/164

This is a follow-up diff from D18720474

We will be releasing a new version of torchvision soon and the signature of those functions is not ready yet, following my comment in https://our.intern.facebook.com/intern/diff/D18720474/?transaction_id=561239541337402



Reviewed By: stephenyan1231

Differential Revision: D20914571

fbshipit-source-id: 1a7560b8f8e46ab42ef376c50b494a4f73923e94
Co-authored-by: Francisco Massa <fmassa@fb.com>

* Add tests for negative samples for Mask R-CNN and Keypoint R-CNN

* Fix lint

* improve stability of test_nms_cuda

This change addresses two issues:

_create_tensors_with_iou() creates test data for the NMS tests. It
takes care to ensure at least one pair of boxes (1st and last) have
IoU around the threshold for the test. However, the constructed
IoU for that pair is _so_ close to the threshold that rounding
differences (presumably) between CPU and CUDA implementations may
result in one suppressing a box in the pair and the other not.
Adjust the construction to ensure the IoU for the box pair is
near the threshold, but far-enough above that both implementations
should agree.

Where 2 boxes have nearly or exactly the same score, the CPU and
CUDA implementations may order them differently. Adjust
test_nms_cuda() to check only that the non-suppressed box lists
include the same members, without regard for ordering.

* adjust assertion in test_nms_cuda

The CPU and CUDA nms implementations each sort the box scores
as part of their work, but the sorts they use are not stable. So
boxes with the same score maybe be processed in opposite order
by the two implmentations.

Relax the assertion in test_nms_cuda (following the model in
pytorch's test_topk()) to allow the test to pass if the output
differences are caused by similarly-scored boxes.

* improve stability of test_nms_cuda

Adjust _create_tensors_with_iou() to ensure we create at least
one box just over threshold that should be suppressed.

* Add CircleCI job for python lint

* Break lint

* Fix

* Fix lint

* Re-enable all tests and remove travis python lint

* Add test for large batches in DeformConv2d

* Clean-up and (try) fix DeformConv2d

* Simplifications and bugfixes

* Try fix CUDA now

* fixes and tests for variable input size

* transform test fix

* Fix comment

* Dynamic shape for keypoint_rcnn

* Update test_onnx.py

* Update rpn.py

* Fix for split on RPN

* Fixes for feedbacks

* flake8

* topk fix

* Fix build

* branch on tracing

* fix for scalar tensor

* Fixes for script type annotations

* Update rpn.py

* clean up

* clean up

* Update rpn.py

* Updated for feedback

* Fix for comments

* revert to use tensor

* Added test for box clip

* Fixes for feedback

* Fix for feedback

* ORT version revert

* Update ort

* Update .travis.yml

* Update test_onnx.py

* Update test_onnx.py

* Tensor sizes

* Fix for dynamic split

* Try disable tests

* pytest verbose

* revert one test

* enable tests

* Update .travis.yml

* Update .travis.yml

* Update .travis.yml

* Update test_onnx.py

* Update .travis.yml

* Passing device

* Fixes for test

* Fix for boxes datatype

* clean up
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

* remove sys.version_info == 2

* remove sys.version_info < 3

* remove from __future__ imports

* remove six from python code

* remove six from setup.py

* remove six from tests

* remove six from references

* remove six from packaging

* revert str to torch._six._string_classes

* revert str to torch._six._string_classes

* Add fill option to RandomPerspective #1972

* Minor fix to docstring syntax

* Add _parse_fill() to get fillcolor (#1972)

* Minor refactoring as per comments.

* Added test for RandomPerspective with fillcolor.

* Force perspective transform in test.

* F.normalize unsqueeze mean & std if necessary

* added tests to F.normalize for 3d mean & std tensors

* modified FasterRCNN to accept negative samples

* remove debug lines

* Change torch.zeros_like to torch.zerros

* Add unit tests

* take the `device` into account
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

* Base decoder for video. (#1747)

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1747

Pull Request resolved: https://github.com/pytorch/vision/pull/1746

Added the implementation of ffmpeg based decoder with functionality that can be used in VUE and TorchVision.

Reviewed By: fmassa

Differential Revision: D19358914

fbshipit-source-id: abb672f89bfaca6351dda2354f0d35cf8e47fa0f

* Integrated base decoder into VideoReader class and video_utils.py (#1766)

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1766

Replaced FfmpegDecoder (incompativle with VUE) by base decoder (compatible with VUE).
Modified python utilities video_utils.py for internal simplification. Public interface got preserved.

Reviewed By: fmassa

Differential Revision: D19415903

fbshipit-source-id: 4d7a0158bd77bac0a18732fe4183fdd9a57f6402

* Optimizating base decoder performance. (#1852)

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1852

Changed base decoder internals for a faster clip processing.

Reviewed By: stephenyan1231

Differential Revision: D19748379

fbshipit-source-id: 58a435f0a0b25545e7bd1a3edb0b1d558176a806

* Minor fix and decoder class members access.

Summary:
Found and fix a bug in cropping algorithm (simple mistyping).
Also derived classes need access to some decoder class members, like initialization parameters - make it protected.

Reviewed By: stephenyan1231, fmassa

Differential Revision: D19895076

fbshipit-source-id: 691336c8e18526b085ae5792ac3546bc387a6db9

* Added missing header for less dependencies. (#1898)

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1898

Include streams/samplers shouldn't depend on decoder headers. Add dependencies directly to the place where they are required.

Reviewed By: stephenyan1231

Differential Revision: D19911404

fbshipit-source-id: ef322a053708405c02cee4562b456b1602fb12fc

* Implemented VUE Asynchronous Decoder

Summary: For Mothership we have found that asynchronous decoder provides a better performance.

Differential Revision: D20026194

fbshipit-source-id: 627b91844b4e3f917002031dd32cb19c239f4ba8

* fix a bug in API read_video_from_memory (#1942)

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1942

In D18720474, it introduces a bug in `read_video_from_memory` API. Thank weiyaowang for reporting it.

Reviewed By: weiyaowang

Differential Revision: D20270179

fbshipit-source-id: 66348c99a5ad1f9129b90e934524ddfaad59de03

* extend decoder to support new video_max_dimension argument (#1924)

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1924

Extend `video reader` decoder python API in Torchvision to support a new argument `video_max_dimension`. This enables the new video decoding use cases. When setting `video_width=0`, `video_height=0`, `video_min_dimension != 0`, and `video_max_dimension != 0`, we can rescale the video clips so that its spatial resolution (height, width) becomes
 - (video_min_dimension, video_max_dimension) if original height < original width
 - (video_max_dimension, video_min_dimension) if original height >= original width

This is useful at video model testing stage, where we perform fully convolution evaluation and take entire video frames without cropping as input. Previously, for instance we can only set `video_width=0`, `video_height=0`, `video_min_dimension = 128`, which will preserve aspect ratio. In production dataset, there are a small number of videos where aspect ratio is either extremely large or small, and when the shorter edge is rescaled to 128, the longer edge is still large. This will easily cause GPU memory OOM when we sample multiple video clips, and put them in a single minibatch.

Now, we can set (for instance) `video_width=0`, `video_height=0`, `video_min_dimension = 128` and `video_max_dimension = 171` so that the rescale resolution is either (128, 171) or (171, 128) depending on whether original height is larger than original width. Thus, we are less likely to have gpu OOM because the spatial size of video clips is determined.

Reviewed By: putivsky

Differential Revision: D20182529

fbshipit-source-id: f9c40afb7590e7c45e6908946597141efa35f57c

* Fixing samplers initialization (#1967)

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1967



No-ops for torchvision diff, which fixes samplers.

Differential Revision: D20397218

fbshipit-source-id: 6dc4d04364f305fbda7ca4f67a25ceecd73d0f20

* Exclude C++ test files
Co-authored-by: Yuri Putivsky <yuri@fb.com>
Co-authored-by: Zhicheng Yan <zyan3@fb.com>

* Fix for roi_align export

* Disable interpolate script module tests

Disable test until export of interpolate script module to ONNX is fixed

* add checkout/assert in roi_pool

* add checkout/assert in roi_align

* move check_roi_boxes_shape func to ops/_utils.py

* add tests

* fix CI

* fix CI
Co-authored-by: Guanheng Zhang <zhangguanheng@devfair0197.h2.fair>

* fix googlenet no aux logits

* small fix
Co-authored-by: eellison <eellison@fb.com>

* Aligned flag in the interfaces

* Aligned flag in the impl, and remove unused comments

* Handling empty bin in forward

* Remove raise error in roi_width

* Aligned flag in the Testcodes

* ci: Add verbosity to pytest results, store in ci

Makes the pytest runs for building conda packages more verbose and
stores the results for viewing inside of CircleCI
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

* test: Skip inception v3 in test_quantized_models

Was causing timeouts on circleci due to long run time, re-enable when
tests can be brought to a reasonable time again.
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

* feat: Added __repr__ attribute to GeneralizedRCNNTransform

Added more details to default __repr__ attribute for printing.

* fix: Put back relative imports

* style: Fixed pep8 compliance

Switched strings with  syntax to f-strings.

* test: Added test for GeneralizedRCNNTransform __repr__

Checked integrity of __repr__ attribute

* test: Fixed unittest for __repr__

Fixed the formatted strings in the __repr__ integrity check for GeneralizedRCNNTransform

* fix: Fixed f-strings for earlier python versions

Switched back f-strings to .format syntax for Python3.5 compatibility.

* fix: Fixed multi-line string

Fixed multiple-line string syntax for compatibility

* fix: Fixed GeneralizedRCNNTransform unittest

Fixed formatting of min_size argument of the resizing part

* Force object annotiation to be an array

* Remove unecessary parentheses

* Change object check

* Remove check for list

* Add test coverage to xml parsing

* Tidy up whitespace

* Fix indentation

* Force object annotiation to be an array

* Remove unecessary parentheses

* Change object check

* Remove check for list

* Add test coverage to xml parsing

* Tidy up whitespace

* torchscriptable functions for video io (#1653)

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1653



created new torchscriptable video io functions as part of the api: read_video_meta_data_from_memory and read_video_from_memory.

Updated the implementation of some of the internal functions to be torchscriptable.

Reviewed By: stephenyan1231

Differential Revision: D18720474

fbshipit-source-id: 4ee646b66afecd2dc338a71fd8f249f25a3263bc

* BugFix
Co-authored-by: Jon Guerin <54725679+jguerin-fb@users.noreply.github.com>

* adding new_empty_tensor symbolic

* flake8

* fix for feedback

* skipping the ORT test

* fix for ORT test

* initial fix

* outsourced num bands lookup

* fix doc

* added pillow version requirement

* simplify number of bands extraction

* remove unrelated change

* remove indirect dependency on pillow>=5.2.0

* extend docstring to transform

* bug fix

* added test