GitLab

Signed-off-by: Jeff Kriske <kriske_jeffery_e@lilly.com>

* Add libpng requirement into conda recipe

* Try to install libjpeg-turbo

* Add PNG reading capabilities

* Remove newline

* Add image extension to compilation instructions

* Include png functions as part of the main library

* Update CMakeLists

* Detect if building on conda-build

* Debug

* More debug messages

* Print globbed libreries

* Print globbed libreries

* Point to correct PNG path

* Remove libJPEG preventively

* Debug extension loading

* Link libpng explicitly

* Link with PNG

* Add PNG reading capabilities

* Add libpng requirement into conda recipe

* Try to install libjpeg-turbo

* Remove newline

* Add image extension to compilation instructions

* Include png functions as part of the main library

* Update CMakeLists

* Detect if building on conda-build

* Debug

* More debug messages

* Print globbed libreries

* Print globbed libreries

* Point to correct PNG path

* Remove libJPEG preventively

* Debug extension loading

* Link libpng explicitly

* Link with PNG

* Install libpng on conda-based wheel distributions

* Add -y flag

* Add -y flag to yum

* Locate LibPNG on windows conda

* Remove empty else

* Copy libpng16.so

* Copy dylib on Mac

* Improve check on Windows

* Try to install ninja using conda on windows

* Use libpng on Windows

* Package lib on windows wheel

* Point library to the correct place

* Include binaries as part of wheel

* Copy libpng.so on linux

* Look for png.h on Windows when using conda-build

* Do not skip png tests on Mac/Win

* Restore libjpeg-turbo

* Install jpeg-turbo on wheel distributions

* Install libjpeg-turbo from conda-forge on wheel distributions

* Do not pull av on conda-build

* Add pillow disclaimer

* Vendors libjpeg-turbo 2.0.4

* Merge JPEG work

* Remove submodules

* Regenerate circle config

* Fix style issues

* Fix C++ style issues

* More style corrections

* Add JPEG-turbo to linking libraries

* More style corrections

* More style corrections

* More style corrections

* Install libjpeg-turbo-devel

* Install libturbo-jpeg on typing pipeline

* Update Circle template

* Windows and Unix turbojpeg have the same linking name

* Install turbojpeg-devel instead of libjpeg-turbo

* Copy TurboJPEG binaries to wheel

* Move test image

* Move back test image

* Update JPEG test path

* Remove dot from extension

* Move image functions to extension

* Use stdout arg in subprocess

* Disable image extension if libpng or turbojpeg are not found

* Append libpng stdout

* Prevent list appending on lists

* Minor path correction

* Minor error correction

* Add linking flags

* Style issues correction

* Address minor review corrections

* Refactor library search

* Restore access index

* Fix JPEG tests

* Update libpng version in Travis

* Add -y flag

* Remove dot

* Update libpng using apt

* Check libpng version

* Change libturbojpeg binary

* Update import

* Change call

* Restore av in conda recipe

* Minor error correction

* Remove unused comment in travis.yml

* Update README

* Fix missing links

* Remove fixes for 16.04

* Remove JPEG-related code

* Remove installation references to turbojpeg

* Remove further references to turbojpeg

* Fix c++ style issues

* Fix c++ style issues

* Fix libpng-config include flag parsing

* Remove conda-forge

* Remove include dirs from main extension

* Do not pass extra include and library paths to main torchvision extension

* Add libpng to environment.yml

* Remove inexistent imports

* Add instructions regarding environment variables to README

* Fix image extension tests

* Add libpng to environment + test fixes

* Minor improvements

* Remove unused Py2 code

* Add stub comments to prevent deletion while merging

* Reintroduce files in order to prevent deletion during merge

* Remove unwanted merge sections

* Restore libpng conda installation on wheel distributions

* Restore comment

* Fix libpng discovery on Windows

* Fix PEP8 style issues

* Add linking flag on Windows

* Remove parenthesis

* Restore libpng during runtime
Co-authored-by: Edgar Andrés Margffoy Tuay <andfoy@gmail.com>
Co-authored-by: Ryad ZENINE <r.zenine@gmail.com>

This reverts commit c1a99b7b.

* Add libpng requirement into conda recipe

* Try to install libjpeg-turbo

* Add PNG reading capabilities

* Remove newline

* Add image extension to compilation instructions

* Include png functions as part of the main library

* Update CMakeLists

* Detect if building on conda-build

* Debug

* More debug messages

* Print globbed libreries

* Print globbed libreries

* Point to correct PNG path

* Remove libJPEG preventively

* Debug extension loading

* Link libpng explicitly

* Link with PNG

* Add PNG reading capabilities

* Add libpng requirement into conda recipe

* Try to install libjpeg-turbo

* Remove newline

* Add image extension to compilation instructions

* Include png functions as part of the main library

* Update CMakeLists

* Detect if building on conda-build

* Debug

* More debug messages

* Print globbed libreries

* Print globbed libreries

* Point to correct PNG path

* Remove libJPEG preventively

* Debug extension loading

* Link libpng explicitly

* Link with PNG

* Install libpng on conda-based wheel distributions

* Add -y flag

* Add -y flag to yum

* Locate LibPNG on windows conda

* Remove empty else

* Copy libpng16.so

* Copy dylib on Mac

* Improve check on Windows

* Try to install ninja using conda on windows

* Use libpng on Windows

* Package lib on windows wheel

* Point library to the correct place

* Include binaries as part of wheel

* Copy libpng.so on linux

* Look for png.h on Windows when using conda-build

* Do not skip png tests on Mac/Win

* Restore libjpeg-turbo

* Install jpeg-turbo on wheel distributions

* Install libjpeg-turbo from conda-forge on wheel distributions

* Do not pull av on conda-build

* Add pillow disclaimer

* Vendors libjpeg-turbo 2.0.4

* Merge JPEG work

* Remove submodules

* Regenerate circle config

* Fix style issues

* Fix C++ style issues

* More style corrections

* Add JPEG-turbo to linking libraries

* More style corrections

* More style corrections

* More style corrections

* Install libjpeg-turbo-devel

* Install libturbo-jpeg on typing pipeline

* Update Circle template

* Windows and Unix turbojpeg have the same linking name

* Install turbojpeg-devel instead of libjpeg-turbo

* Copy TurboJPEG binaries to wheel

* Move test image

* Move back test image

* Update JPEG test path

* Remove dot from extension

* Move image functions to extension

* Use stdout arg in subprocess

* Disable image extension if libpng or turbojpeg are not found

* Append libpng stdout

* Prevent list appending on lists

* Minor path correction

* Minor error correction

* Add linking flags

* Style issues correction

* Address minor review corrections

* Refactor library search

* Restore access index

* Fix JPEG tests

* Update libpng version in Travis

* Add -y flag

* Remove dot

* Update libpng using apt

* Check libpng version

* Change libturbojpeg binary

* Update import

* Change call

* Restore av in conda recipe

* Minor error correction

* Remove unused comment in travis.yml

* Update README

* Fix missing links

* Remove fixes for 16.04

* Remove JPEG-related code

* Remove installation references to turbojpeg

* Remove further references to turbojpeg

* Fix c++ style issues

* Fix c++ style issues

* Fix libpng-config include flag parsing

* Remove conda-forge

* Remove include dirs from main extension

* Do not pass extra include and library paths to main torchvision extension

* Add libpng to environment.yml

* Remove inexistent imports

* Add instructions regarding environment variables to README
Co-authored-by: Ryad ZENINE <r.zenine@gmail.com>

This reverts commit 766721b1.

* Add libpng requirement into conda recipe

* Try to install libjpeg-turbo

* Add PNG reading capabilities

* Remove newline

* Add image extension to compilation instructions

* Include png functions as part of the main library

* Update CMakeLists

* Detect if building on conda-build

* Debug

* More debug messages

* Print globbed libreries

* Print globbed libreries

* Point to correct PNG path

* Remove libJPEG preventively

* Debug extension loading

* Link libpng explicitly

* Link with PNG

* Add PNG reading capabilities

* Add libpng requirement into conda recipe

* Try to install libjpeg-turbo

* Remove newline

* Add image extension to compilation instructions

* Include png functions as part of the main library

* Update CMakeLists

* Detect if building on conda-build

* Debug

* More debug messages

* Print globbed libreries

* Print globbed libreries

* Point to correct PNG path

* Remove libJPEG preventively

* Debug extension loading

* Link libpng explicitly

* Link with PNG

* Install libpng on conda-based wheel distributions

* Add -y flag

* Add -y flag to yum

* Locate LibPNG on windows conda

* Remove empty else

* Copy libpng16.so

* Copy dylib on Mac

* Improve check on Windows

* Try to install ninja using conda on windows

* Use libpng on Windows

* Package lib on windows wheel

* Point library to the correct place

* Include binaries as part of wheel

* Copy libpng.so on linux

* Look for png.h on Windows when using conda-build

* Do not skip png tests on Mac/Win

* Restore libjpeg-turbo

* Install jpeg-turbo on wheel distributions

* Install libjpeg-turbo from conda-forge on wheel distributions

* Do not pull av on conda-build

* Add pillow disclaimer

* Vendors libjpeg-turbo 2.0.4

* Merge JPEG work

* Remove submodules

* Regenerate circle config

* Fix style issues

* Fix C++ style issues

* More style corrections

* Add JPEG-turbo to linking libraries

* More style corrections

* More style corrections

* More style corrections

* Install libjpeg-turbo-devel

* Install libturbo-jpeg on typing pipeline

* Update Circle template

* Windows and Unix turbojpeg have the same linking name

* Install turbojpeg-devel instead of libjpeg-turbo

* Copy TurboJPEG binaries to wheel

* Move test image

* Move back test image

* Update JPEG test path

* Remove dot from extension

* Move image functions to extension

* Use stdout arg in subprocess

* Disable image extension if libpng or turbojpeg are not found

* Append libpng stdout

* Prevent list appending on lists

* Minor path correction

* Minor error correction

* Add linking flags

* Style issues correction

* Address minor review corrections

* Refactor library search

* Restore access index

* Fix JPEG tests

* Update libpng version in Travis

* Add -y flag

* Remove dot

* Update libpng using apt

* Check libpng version

* Change libturbojpeg binary

* Update import

* Change call

* Restore av in conda recipe

* Minor error correction

* Remove unused comment in travis.yml

* Update README

* Fix missing links

* Remove fixes for 16.04
Co-authored-by: Ryad ZENINE <r.zenine@gmail.com>

* Switch torchvision registrations to new operator registration API.

This is still registering everything as catchalls, so we're really just
moving deck chairs around, but payoff is coming soon.
Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Port roi_align to actually use dispatcher
Signed-off-by: Edward Z. Yang <ezyang@fb.com>

Summary:
Allow writes of >= 2^32 bytes. High-res video can cross this threshold sometimes.
LHS is `size_t`, but RHS is all `int32`, and will overflow for output tensors >2Gb.

Reviewed By: jsawruk

Differential Revision: D21255664

fbshipit-source-id: 7b4c5da989777297a89e73615aaeee8c7a13186a
Co-authored-by: Tilak Sharma <tilaksharma@fb.com>

* Add more tests to NMS

* Fix lint

* Avoid `using` in header files

* Fix clang_format

* use clang-format-7 to reformat code

* Fix missing include for OSX in video decoder

* clang-format

Fixes https://github.com/pytorch/vision/issues/2214#issuecomment-628636663

I don't know why the building is not working with `--expt-relaxed-constexpr` flag set, but it is generally a good idea to declare this as `__host__ __device__`

Fix https://github.com/pytorch/vision/issues/2193.

* Don't include CUDAApplyUtils.cuh

* fix format

* fix atomic

* fix the use of contiguous() in kernels

* clang-format

* add a contiguous in nms
Co-authored-by: Yuxin Wu <ppwwyyxx@users.noreply.github.com>

Torchvision includes at least 3 bits of code that calculate
box Intersection over Union values (and usually compare to
a threshold):

- box_iou in torchvision/ops/boxes.py
- devIoU in torchvision/csrc/cuda/nms_cuda.cu
- nms_cpu_kernel in torchvision/csrc/cpu/nms_cpu.cpp

The calculations were performed slightly differently between
those, leading to occasional differences in results.

Update devIoU to use the same method as the others for better
consistency.

This change improves agreement between the CPU and CUDA
calculations but the results can still differ slightly.
Setting NVCC_FLAGS to include "--fmad=true" would provide
still better agreement, but with likely cost to performance.

* Replace **.is_cuda() to just is_cuda()

* Replace type to scalar_type

* Fix lint, clang-format

* Fix lint, clang-format

* Add clang-format to CircleCI

* Fix for clang-format version

* Fix lint and remove Travis CI

* Seeing if lost commit comes back

* Fix lint

* Re-enable all tests

* Add test for large batches in DeformConv2d

* Clean-up and (try) fix DeformConv2d

* Simplifications and bugfixes

* Try fix CUDA now

* fix shape error for deform conv gpu op

recover shape of columns for next iteration in for loops, previous version will cause error when batch_sz / n_parallel_imgs > 1

* fix shape error for deform conv cpu op

recover shape of columns for next iteration in for loops, previous version will cause error when batch_sz / n_parallel_imgs > 1

* Fix C++ lint

* More fixes

* Base decoder for video. (#1747)

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1747

Pull Request resolved: https://github.com/pytorch/vision/pull/1746

Added the implementation of ffmpeg based decoder with functionality that can be used in VUE and TorchVision.

Reviewed By: fmassa

Differential Revision: D19358914

fbshipit-source-id: abb672f89bfaca6351dda2354f0d35cf8e47fa0f

* Integrated base decoder into VideoReader class and video_utils.py (#1766)

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1766

Replaced FfmpegDecoder (incompativle with VUE) by base decoder (compatible with VUE).
Modified python utilities video_utils.py for internal simplification. Public interface got preserved.

Reviewed By: fmassa

Differential Revision: D19415903

fbshipit-source-id: 4d7a0158bd77bac0a18732fe4183fdd9a57f6402

* Optimizating base decoder performance. (#1852)

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1852

Changed base decoder internals for a faster clip processing.

Reviewed By: stephenyan1231

Differential Revision: D19748379

fbshipit-source-id: 58a435f0a0b25545e7bd1a3edb0b1d558176a806

* Minor fix and decoder class members access.

Summary:
Found and fix a bug in cropping algorithm (simple mistyping).
Also derived classes need access to some decoder class members, like initialization parameters - make it protected.

Reviewed By: stephenyan1231, fmassa

Differential Revision: D19895076

fbshipit-source-id: 691336c8e18526b085ae5792ac3546bc387a6db9

* Added missing header for less dependencies. (#1898)

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1898

Include streams/samplers shouldn't depend on decoder headers. Add dependencies directly to the place where they are required.

Reviewed By: stephenyan1231

Differential Revision: D19911404

fbshipit-source-id: ef322a053708405c02cee4562b456b1602fb12fc

* Implemented VUE Asynchronous Decoder

Summary: For Mothership we have found that asynchronous decoder provides a better performance.

Differential Revision: D20026194

fbshipit-source-id: 627b91844b4e3f917002031dd32cb19c239f4ba8

* fix a bug in API read_video_from_memory (#1942)

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1942

In D18720474, it introduces a bug in `read_video_from_memory` API. Thank weiyaowang for reporting it.

Reviewed By: weiyaowang

Differential Revision: D20270179

fbshipit-source-id: 66348c99a5ad1f9129b90e934524ddfaad59de03

* extend decoder to support new video_max_dimension argument (#1924)

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1924

Extend `video reader` decoder python API in Torchvision to support a new argument `video_max_dimension`. This enables the new video decoding use cases. When setting `video_width=0`, `video_height=0`, `video_min_dimension != 0`, and `video_max_dimension != 0`, we can rescale the video clips so that its spatial resolution (height, width) becomes
 - (video_min_dimension, video_max_dimension) if original height < original width
 - (video_max_dimension, video_min_dimension) if original height >= original width

This is useful at video model testing stage, where we perform fully convolution evaluation and take entire video frames without cropping as input. Previously, for instance we can only set `video_width=0`, `video_height=0`, `video_min_dimension = 128`, which will preserve aspect ratio. In production dataset, there are a small number of videos where aspect ratio is either extremely large or small, and when the shorter edge is rescaled to 128, the longer edge is still large. This will easily cause GPU memory OOM when we sample multiple video clips, and put them in a single minibatch.

Now, we can set (for instance) `video_width=0`, `video_height=0`, `video_min_dimension = 128` and `video_max_dimension = 171` so that the rescale resolution is either (128, 171) or (171, 128) depending on whether original height is larger than original width. Thus, we are less likely to have gpu OOM because the spatial size of video clips is determined.

Reviewed By: putivsky

Differential Revision: D20182529

fbshipit-source-id: f9c40afb7590e7c45e6908946597141efa35f57c

* Fixing samplers initialization (#1967)

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1967



No-ops for torchvision diff, which fixes samplers.

Differential Revision: D20397218

fbshipit-source-id: 6dc4d04364f305fbda7ca4f67a25ceecd73d0f20

* Exclude C++ test files
Co-authored-by: Yuri Putivsky <yuri@fb.com>
Co-authored-by: Zhicheng Yan <zyan3@fb.com>

* Added code to support creating extension on ROCm

* max -> fmaxf conversion for hipification

* added WITH_HIP flag for hipExtension

* added appropriate headers for HIP build

* use USE_ROCM in condition to build

* change fmaxf and fminf calls

* fminf -> min

* fix the check for ROCM_HOME

* more robust checking for rocm pytorch

* add check for pytorch version before using HIP extensions

* conditional reading of ROCM_HOME

* Aligned flag in the interfaces

* Aligned flag in the impl, and remove unused comments

* Handling empty bin in forward

* Remove raise error in roi_width

* Aligned flag in the Testcodes

This reverts commit 28b7f8ae.

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1747

Pull Request resolved: https://github.com/pytorch/vision/pull/1746



Added the implementation of ffmpeg based decoder with functionality that can be used in VUE and TorchVision.

Reviewed By: fmassa

Differential Revision: D19358914

fbshipit-source-id: abb672f89bfaca6351dda2354f0d35cf8e47fa0f
Co-authored-by: Yuri Putivsky <yuri@fb.com>

1. Let the IOU function compare with threshold. This avoid a division. Similar strategy is also used in https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/kernels/non_max_suppression_op.cu.cc
2. Only compute the upper triangle of the mask.

This speeds up the kernel about 20% (tested on GTX 1080Ti, with 20 input cases dumped from a Mask R-CNN inference job).

* Add Deformable Convolution operation.

This adds the deformable convolution operation, as described in Deformable Convolutional Networks (https://arxiv.org/abs/1703.06211).

- The code is based on https://github.com/open-mmlab/mmdetection/blob/master/mmdet/ops/dcn/src/deform_conv_cuda.cpp ; the whole code was modified and refactored to remove redundancies and increase clarity, and to adapt it to torchvision.

- The CPU part is a direct copy of the CUDA code; it might make sense to do follow-up adjustments in the CPU code to simplify it / optimize it, or to reuse functionality between CPU and CUDA..

- We also add tests (with a non-trivial set of parameters); they can be made more robust by randomizing the parameters and executing multiple times.

* Update DeformConv to be more consistent w/ Conv2d

* rename some variables and arguments to match Conv2d;
* add optional bias;
* add weight, offset and bias as module parameters;
* remove the n_parallel_imgs parameter;
* Fix __repr__;
* etc..

Initialization of weight and bias is the same as in Conv2d, and
initialization of offsets to zero is the same as in the paper.

This also includes some other small unrelated fixes/improvements.

* Apply clang-format in DeformConv files.

* Import Optional type annotation

* Remove offset param from DeformConv2d module

- We pass the offset in the forward of DeformConv2d, instead of having
an internal parameter. This adds some complexity to creating the module
(e.g. now you have to worry about the output size, to create the
offset), but it gives more flexibility.
- We also use make_tuple for tuple creation, in an attempt to fix error
w/ older compilers.

* Replace abs by std::abs

Old gcc versions were giving wrong results here, because they would
resolve abs as int -> int, thus causing undesired truncation. Replacing
abs by std::abs should allow for correct overloading of abs as float -> float.

* Reorder declarations for clarity

* Reorder weight and offset args in deform_conv2d

We place offset arg before the weight arg, to be more
consistent with DeformConv2d.forward(input, offset)

* Replace abs by std::abs in DeformConv_cuda

* almost working...

* respond to comments

* add empty tensor op, handle different output types in generalized rcnn

* clean ups

* address comments

* more changes

* it's working!

* torchscript bugs

* add script/ eager test

* eval script model

* fix flake

* division import

* py2 compat

* update test, fix arange bug

* import division statement

* fix linter

* fixes

* changes needed for JIT master

* cleanups

* remove imagelist_to

* requested changes

* Make FPN backwards-compatible and torchscript compatible

We remove support for feature channels=0, but support for it was already a bit limited

* Fix ONNX regression

Rename with_bias() to bias(), and output_channels() to out_channels() in C++ conv layer options usage (#1576)