GitLab

* 'make_dataset' as staticmethod of 'DatasetFolder'

* a better fix
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

* Add MobileNetV3 Architecture in TorchVision (#3182)

* Adding implementation of network architecture

* Adding rmsprop support on the train.py

* Adding auto-augment and random-erase in the training scripts.

* Adding support for reduced tail on MobileNetV3.

* Tagging blocks with comments.

* Adding documentation, pre-trained model URL and a minor refactoring.

* Handling better untrained supported models.

* Introduce small score threshold on rpn

* Adding docs and fixing keypoint and mask.

* Making value 0.0 by default for BC.

* Fixing for onnx.

* Update threshold.

* Removing non-default threshold from reference scripts.
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

* initial commit of widerface dataset

* comment out old code

* improve parsing of annotation files

* code cleanup and fix docstring comments

* speed up check for quota exceeded

* cleanup print statements

* reformat code and remove print statements

* minor code cleanup and reformatting

* add more comments

* reuse variable

* reverse formatting changes

* fix flake8 errors

* add type annotations

* fix mypy errors

* add a base_folder to root directory

* some formatting fixes

* GDrive threshold does not throw 403 error

* testing new download logic

* cleanup logic for download and integrity check

* use a better variable name

* format fix

* reorder list in docstring

* initial widerface unit test - fails on MD5 check

* use list of dictionaries to store dataset

* fix docstring formatting

* remove unnecessary error checking

* fix type checker error

* revert typo fix

* rename var constants, use file context manager, verify str args

* fix flake8 error

* fix checking target_type argument values

* create uncompressed dataset folders

* cleanup unit tests for widerface

* use correct os function

* add more info to docstring

* disable unittests for windows

* fix _check_integrity logic

* update docstring

* remove citation

* remove target_type option

* fix formatting issue
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* remove comment and add more info to docstring

* update type annotations

* restart CI jobs
Co-authored-by: Joshua Bradley <jgbrad3@evoforge.org>
Co-authored-by: Philip Meier <github.pmeier@posteo.de>
Co-authored-by: vfdev <vfdev.5@gmail.com>

* Fixing trainable_layers bug.

* minor doc fixes

* remove unused imports after manual review

* Update torchvision/datasets/voc.py
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

* remove two more instances
Co-authored-by: Ben Weinstein <benweinstein@Bens-MacBook-Pro.local>
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

* adjust_hue

* adjust_*

* colorjitter

* Exposing LevelMapper params in MultiScaleRoIAlign

Ability to set the `canonical_scale` and `canonical_level` of LevelMapper from the init method of MultiScaleRoIAlign. 
Not sure if this is the right place to expose those parameters. 
Also, additional tests should be written.

Discussion in issue #3129

* MultiScaleRoIAlign keyword-only parameters + docs (pytorch#3129)

* Linter fix (trailing whitespace)
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

* Adding TORCH_SELECTIVE_* macros on op registration.

* Adding torchvision namespace.

* Initial doc clean-up

* Remove all private docs

* Rename files

* Highlight backend inconsistencies

* Sequence and number

* [Need checking] AutoAugment related doc change

* Revert name changes

* Patches required for FBCode merge.

* Patching quantization model imports.

* import QuantizableMobileNetV2

* Adding newline to avoid lit errors

Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

* Update ImageReadMode error messages, add newline at the end of image_read_mode.h, replace define with const in image_read_mode.h, add documentation to ImageReadMode enum

* Update readpng_cpu and readjpeg_cpu error messages

* Update image.py documentation

* Moving mobilenet.py to mobilenetv2.py

* Adding mobilenet.py for BC.

* Extending ConvBNReLU for reuse.

* Reduce import scope on mobilenet to only the public and versioned classes and methods.

* Fixing incorrect doc example in MNASNet.

* Fixing incorrect output.

* added the helper method for dimension checks

* unit tests for dimensio check function in functional_tensor

* code formatting and typing

* moved torch image check after tensor check

* unit testcases for test_assert_image_tensor added and refactored

* separate unit testcase file deleted

* assert_image_tensor added to newly created 6 methods

* test cases added for new 6 mthohds

* removed wrongly pasted posterize method and added solarize method for testing
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

* Replacing all torch.jit.annotations with typing

* Replacing remaining typing

* Invert Transform (#3104)

* Adding invert operator.

* Make use of the _assert_channels().

* Update upper bound value.

* Remove private doc from invert, create or reuse generic testing methods to avoid duplication of code in the tests. (#3106)

* Create posterize transformation and refactor common methods to assist reuse. (#3108)

* Implement the solarize transform. (#3112)

* Implement the adjust_sharpness transform (#3114)

* Adding functional operator for sharpness.

* Adding transforms for sharpness.

* Handling tiny images and adding a test.

* Implement the autocontrast transform. (#3117)

* Implement the equalize transform (#3119)

* Implement the equalize transform.

* Turn off deterministic for histogram.

* Fixing test. (#3126)

* Force ratio to be float to avoid numeric overflows on blend. (#3127)

* Separate the tests of Adjust Sharpness from ColorJitter. (#3128)

* Add AutoAugment Policies and main Transform (#3142)

* Separate the tests of Adjust Sharpness from ColorJitter.

* Initial implementation, not-jitable.

* AutoAugment passing JIT.

* Adding tests/docs, changing formatting.

* Update test.

* Fix formats

* Fix documentation and imports.

* Apply changes from code review:
- Move the transformations outside of AutoAugment on a separate method.
- Renamed degenerate method for sharpness for better clarity.

* Update torchvision/transforms/functional.py
Co-authored-by: vfdev <vfdev.5@gmail.com>

* Apply more changes from code review:
- Add InterpolationMode parameter.
- Move all declarations away from AutoAugment constructor and into the private method.

* Update documentation.

* Apply suggestions from code review
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

* Apply changes from code review:
- Refactor code to eliminate as any to() and clamp() as possible.
- Reuse methods where possible.
- Apply speed ups.

* Replacing pad.
Co-authored-by: vfdev <vfdev.5@gmail.com>
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

* Removing VISION_API from backward() methods and adding a ops.h

* Fixing clang format.

* Fix an issue that ShuffleNetV2 model is exported to a wrong ONNX file if dynamic_axes field was provided.

* Add a ut for the bug fix.

* Fix flake8 issue.

* Don't access each element in x.shape, use x.size() instead.
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

Summary:
* Reduce unnecessary header inclusions in models and io.

* Move autocast to separate folder and hide autograd implementation in an anonymous namespace.

* Moving files in subfolders.

Reviewed By: fmassa

Differential Revision: D25461523

fbshipit-source-id: 756eeb6848aacaa474de4825ed4c1045d17e2cea

* Moving deform_conv2d op registration.

* Moving nms op registration.

* Moving new_empty_tensor op registration.

* Moving ps_roi_align op registration.

* Moving ps_roi_pool op registration.

* Moving roi_align op registration.

* Moving roi_pool op registration.

* Restoring headers for forward/backward and fixing styles.

* Restoring the test hack on windows.

* Stricter header inclusion.

This has been fixed in PyTorch in https://github.com/pytorch/pytorch/pull/40897

accimage always stores images as uint8, so let's be compatible with the internal representation. Tests were failing without this as it would return a float32 image normalized between 0-1

We could have errors such as aten/src/ATen/native/cpu/PowKernel.cpp:41:5:  runtime error: 5.7896e+76 is outside the range of representable values of type 'float'

* Fixing upperbound value on tests and documentation.

* Limit the number of channels on adjust_* transoforms.

* Encapsulate and standardize deform_conv2d (#3074)

* Rename files.

* Standardizing method names.

* Adding anonymous namespaces.

* Applying C++ naming rules and alinging variable names across headers and cpp files.

* Syncing names across implementations.

* Rename deform_conv2d.h to deform_conv2d.cpp

* Use header files:
- Create header files for kernel implementation and remove definitions from vision_*.h files.
- Eliminate unnecessary headers and ensure all cpp include their headers.

* Change the naming convention for kernel implementations.

* Remove the _param postfix from the variables and standardizing names.

* Exposing public forward/backward methods to the C++ API and moving methods around to minimize git blame changes.

* Encapsulate and standardize nms (#3081)

* Syncing, where possible, the names of functions across devices.

* Adding all internal functions in anonymous namespaces.

* Renaming C++/CUDA kernel files and moving operator code from header to cpp file.

* Create foreach cpp file a separate header file with "public" functions.

* Removing unnecessary repeated includes.

* Update CMakeLists.txt to include all headers.

* Encapsulate and standardize ps_roi_align (#3082)

* Renaming C++ files & methods according to recommended naming conventions and aligning them with Python's API.
Syncing, where possible, the names of functions across devices.

* Adding all internal functions in anonymous namespaces.

* Renaming C++/CUDA kernel files and moving operator code from header to cpp file.

* Create foreach cpp file a separate header file with "public" functions.

* Removing unnecessary repeated includes.

* Encapsulate and standardize ps_roi_pool (#3084)

* Renaming C++ files & methods according to recommended naming conventions and aligning them with Python's API.

* Adding all internal functions in anonymous namespaces.

* Renaming C++/CUDA kernel files and moving operator code from header to cpp file.

* Create foreach cpp file a separate header file with "public" functions.

* Removing unnecessary repeated includes.

* Encapsulate and standardize roi_align (#3085)

* Renaming C++ files & methods according to recommended naming conventions and aligning them with Python's API.

* Adding all internal functions in anonymous namespaces.

* Renaming C++/CUDA kernel files and moving operator code from header to cpp file.

* Create foreach cpp file a separate header file with "public" functions.

* Removing unnecessary repeated includes.

* Encapsulate and standardize roi_pool  (#3088)

* Renaming C++ files & methods according to recommended naming conventions and aligning them with Python's API.

* Adding all internal functions in anonymous namespaces.

* Syncing variable names between the cpp files and their header files.

* Renaming C++/CUDA kernel files and moving operator code from header to cpp file.

* Create foreach cpp file a separate header file with "public" functions.

* Removing unnecessary repeated includes.

* Encapsulate and standardize new_empty_tensor_op (#3089)

* Renaming C++ files & methods according to recommended naming conventions and aligning them with Python's API.

* Create foreach cpp file a separate header file with "public" functions.

* Adding all internal functions in anonymous namespaces.

* Convert to const ref all possible parameters.

* Removing unnecessary repeated includes.

* Encapsulate and standardize C++ Ops - Clean up (#3094)

* Removing unnecessary repeated includes.

* Remove unnecessary vision_cpu.h, vision_cuda.h, autocast.h.

* Fixing naming convention and correcting method names on macros.

* Turn on clang formatter for cu files and fixing broken styles.

* Replace "#ifndef ... #define ... #endif" with "#pragma once" on header files.

* Adding operator methods in vision::ops namespace. (#3096)

* Adding operator methods in vision::ops namespace.

* Replace general.h with macros.h

* Adding vision.h to the necessary cpp files.

Replace tabs with spaces, add newlines to files and replace whitelist with allowlist

Operator registration of functions with noexcept doesn't work on some compilers, see https://github.com/pytorch/pytorch/issues/48667

* Fill color support for tensor affine transforms

* PEP fix

* Docstring changes and float support

* Docstring update for transforms and float type cast

* Cast only for Tensor

* Temporary patch for lack of Union type support, plus an extra unit test

* More plausible bilinear filling for tensors

* Keep things simple & New docstrings

* Fix lint and other issues after merge

* make it in one line

* Docstring and some code modifications

* More tests and corresponding changes for transoforms and docstring changes

* Simplify test configs

* Update test_functional_tensor.py

* Update test_functional_tensor.py

* Move assertions
Co-authored-by: vfdev <vfdev.5@gmail.com>

* add UUID in LOG() in decoder

* Fix lint

* More lint

* concatenate small tensors into big ones to reduce the use of shared file descriptor (#1694)

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1694



- PT dataloader forks worker process to speed up the fetching of dataset example.  The recommended way of multiprocess context is `forkserver` rather than `fork`.

- Main process and worker processes will share the dataset class instance, which avoid duplicating the dataset and save memory. In this process, `ForkPickler(..).dumps(...)` will be called to serialize the objects, including objects within dataset instance recursively. `VideoClips` instance internally uses O(N) `torch.Tensor` to store per-video information, such as pts, and possible clips, where N is the No. of videos.

- During dumping, each `torch.Tensor` will use one File Descriptor (FD). The OS default max limit of FD is 65K by using `ulimit -n` to query. The number of tensors in `VideoClips` often exceeds the limit.

- To resolve this issue, we use a few big tensors by concatenating small tensors in the `__getstate__()` method, which will be called during pickling. This will only require O(1) tensors.

- When this diff is landed, we can abondon D19173248

In D19173397, in ClassyVision, we change the mp context from `fork` to `forkserver`, and finally can run the PT dataloader without hanging issues.

Reviewed By: fmassa

Differential Revision: D19179991

fbshipit-source-id: c8716775c7c154aa33d93b25d112d2a59ea688a9

* Try to fix Windows

* Try fix Windows v2

* Disable tests on Windows

* Add back necessary part

* Try fix OSX (and maybe Windows)

* Fix

* Try enabling Windows
Co-authored-by: Zhicheng Yan <zyan3@fb.com>

* Fix: Improve the bounding boxes implementation

Use write_png instead of PIL in test_draw_boxes()
Initialize txt_font only once

* Remove channels permutation in test_draw_boxes
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>
Co-authored-by: Aditya Khursale <akhursale@nvidia.com>
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>