GitLab

* [FEAT] Start implementing Food101 using the new datasets API. WIP.

* [FEAT] Generate Food101 categories and start the test mock.

* [FEAT] food101 dataset code seems to work now.

* [TEST] food101 mock update.

* [FIX] Some fixes thanks to running food101 tests.

* [FIX] Fix mypy checks for the food101 file.

* [FIX] Remove unused numpy.

* [FIX] Some changes thanks to code review.

* [ENH] More idomatic dataset code thanks to code review.

* [FIX] Remove unused cast.

* [ENH] Set decompress and extract to True for some performance gains.

* [FEAT] Use the preprocess=decompress keyword.

* [ENH] Use the train and test.txt file instead of the .json variants and simplify code + update mock data.

* [ENH] Better food101 mock data generation.

* [FIX] Remove a useless print.
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* fix PCAM prototype dataset

* update Zip and Tar archive loader datapipes

* only fail on warnings from the pytorch ecosystem

* Revert "only fail on warnings from the pytorch ecosystem"

This reverts commit 2bf3aa6f2d875a4055f7f3ed0b468316fc60d4f4.

* added usps dataset

* fixed type issues

* fix mobilnet norm layer test (#5643)

* xfail mobilnet norm layer test

* fix test

* More robust check in tests for 16 bits images (#5652)

* Prefer nvidia channel for conda builds (#5648)

To mitigate missing `libcupti.so` dependency

* fix torchdata CI installation (#5657)

* update urls for kinetics dataset (#5578)

* update urls for kinetics dataset

* update urls for kinetics dataset

* remove errors

* update the changes and add test option to split

* added test to valid values for split arg

* change .txt to .csv for annotation url of k600
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>

* Port Multi-weight support from prototype to main (#5618)

* Moving basefiles outside of prototype and porting Alexnet, ConvNext, Densenet and EfficientNet.

* Porting googlenet

* Porting inception

* Porting mnasnet

* Porting mobilenetv2

* Porting mobilenetv3

* Porting regnet

* Porting resnet

* Porting shufflenetv2

* Porting squeezenet

* Porting vgg

* Porting vit

* Fix docstrings

* Fixing imports

* Adding missing import

* Fix mobilenet imports

* Fix tests

* Fix prototype tests

* Exclude get_weight from models on test

* Fix init files

* Porting googlenet

* Porting inception

* porting mobilenetv2

* porting mobilenetv3

* porting resnet

* porting shufflenetv2

* Fix test and linter

* Fixing docs.

* Porting Detection models (#5617)

* fix inits

* fix docs

* Port faster_rcnn

* Port fcos

* Port keypoint_rcnn

* Port mask_rcnn

* Port retinanet

* Port ssd

* Port ssdlite

* Fix linter

* Fixing tests

* Fixing tests

* Fixing vgg test

* Porting Optical Flow, Segmentation, Video models (#5619)

* Porting raft

* Porting video resnet

* Porting deeplabv3

* Porting fcn and lraspp

* Fixing the tests and linter

* Porting docs, examples, tutorials and galleries (#5620)

* Fix examples, tutorials and gallery

* Update gallery/plot_optical_flow.py
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>

* Fix import

* Revert hardcoded normalization

* fix uncommitted changes

* Fix bug

* Fix more bugs

* Making resize optional for segmentation

* Fixing preset

* Fix mypy

* Fixing documentation strings

* Fix flake8

* minor refactoring
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>

* Resolve conflict

* Porting model tests (#5622)

* Porting tests

* Remove unnecessary variable

* Fix linter

* Move prototype to extended tests

* Fix download models job

* Update CI on Multiweight branch to use the new weight download approach (#5628)

* port Pad to prototype transforms (#5621)

* port Pad to prototype transforms

* use literal

* Bump up LibTorchvision version number for Podspec to release Cocoapods (#5624)
Co-authored-by: Anton Thomma <anton@pri.co.nz>
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

* pre-download model weights in CI docs build (#5625)

* pre-download model weights in CI docs build

* move changes into template

* change docs image

* Regenerated config.yml
Co-authored-by: Philip Meier <github.pmeier@posteo.de>
Co-authored-by: Anton Thomma <11010310+thommaa@users.noreply.github.com>
Co-authored-by: Anton Thomma <anton@pri.co.nz>

* Porting reference scripts and updating presets (#5629)

* Making _preset.py classes

* Remove support of targets on presets.

* Rewriting the video preset

* Adding tests to check that the bundled transforms are JIT scriptable

* Rename all presets from *Eval to *Inference

* Minor refactoring

* Remove --prototype and --pretrained from reference scripts

* remove  pretained_backbone refs

* Corrections and simplifications

* Fixing bug

* Fixing linter

* Fix flake8

* restore documentation example

* minor fixes

* fix optical flow missing param

* Fixing commands

* Adding weights_backbone support in detection and segmentation

* Updating the commands for InceptionV3

* Setting `weights_backbone` to its fully BC value (#5653)

* Replace default `weights_backbone=None` with its BC values.

* Fixing tests

* Fix linter

* Update docs.

* Update preprocessing on reference scripts.

* Change qat/ptq to their full values.

* Refactoring preprocessing

* Fix video preset

* No initialization on VGG if pretrained

* Fix warning messages for backbone utils.

* Adding star to all preset constructors.

* Fix mypy.
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>
Co-authored-by: Philip Meier <github.pmeier@posteo.de>
Co-authored-by: Anton Thomma <11010310+thommaa@users.noreply.github.com>
Co-authored-by: Anton Thomma <anton@pri.co.nz>

* Apply suggestions from code review
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* use decompressor for extracting bz2

* Apply suggestions from code review
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* Apply suggestions from code review
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* fixed lint fails

* added tests for USPS

* check image shape

* fix tests

* check shape on image directly

* Apply suggestions from code review
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* removed test and comments

* Update test/test_prototype_builtin_datasets.py
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>
Co-authored-by: Philip Meier <github.pmeier@posteo.de>
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>
Co-authored-by: Nikita Shulga <nshulga@fb.com>
Co-authored-by: Sahil Goyal <sahilgoyal6801@gmail.com>
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>
Co-authored-by: Anton Thomma <11010310+thommaa@users.noreply.github.com>
Co-authored-by: Anton Thomma <anton@pri.co.nz>

* Moving `sequence_to_str` to `torchvision._utils`

* Fix linter

* Rename test_prototype_utils test to test_internal_utils

* [DEBUG] eurosat prototype on windows and macos

* print paths

* fix eurosat mock data setup

* revert changes

* minor cleanup

* basic stanford cars structure added.

* added class in the init.

* how to decode the images

* fix typo in init

* works partially

* formatting

* formatting

* formatting

* formatting

* formatting

* minor changes

* minor changes

* issue resolved

* both datapipes are in sync

* checksum dict keys changed

* formatting

* "ufmt formatting"

* "ufmt formatting"

* formatting

* minor change

* minor change

* test added

* lint (end of line issue)

* lint (end of line issue)

* minor changes

* mock dataset formatting

* minor changes

* hopefully final

* formatting

* prototype test

* prototype test

* prototype test

* prototype test

* prototype test

* prototype test

* prototype test

* prototype test

* Update builtin_dataset_mocks.py

* Update stanford_cars.py

* Update torchvision/prototype/datasets/_builtin/stanford_cars.py
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* Update torchvision/prototype/datasets/_builtin/stanford_cars.py
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* Update torchvision/prototype/datasets/_builtin/stanford_cars.py
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* Update torchvision/prototype/datasets/_builtin/stanford_cars.py
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* formatting and cleaning

* formatting and cleaning

* Update torchvision/prototype/datasets/_builtin/stanford_cars.py
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* generate catefories script

* generate categories script added

* sigh

* sigh

* sigh

* minor changes

* linting
Co-authored-by: Philip Meier <github.pmeier@posteo.de>
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>

* add eurosat

* revert formatting

* port test and make style changes

* add eurosat to __init__

* fix pathlib error

* create dataset zipfile and revert pre commit changes

* remove unecessary variable in resources

* revert auto formatter changes and modify ufmt version

* revert change to contributing guide
Co-authored-by: Dbhasin1 <drishti_b@me.iitr.c.in>
Co-authored-by: Nicolas Hug <nicolashug@fb.com>

* add country211

* remove unused import

* map val to valid and use path comparator

* remove unused import

* resolve keyerror

* map split names in dataset mock
Co-authored-by: Dbhasin1 <drishti_b@me.iitr.c.in>
Co-authored-by: Nicolas Hug <nicolashug@fb.com>

* revamp prototype features (#5283)

* remove decoding from prototype datasets (#5287)

* remove decoder from prototype datasets

* remove unused imports

* cleanup

* fix readme

* use OneHotLabel in SEMEION

* improve voc implementation

* revert unrelated changes

* fix semeion mock data

* fix pcam

* readd functional transforms API to prototype (#5295)

* readd functional transforms

* cleanup

* add missing imports

* remove __torch_function__ dispatch

* readd repr

* readd empty line

* add test for scriptability

* remove function copy

* change import from functional tensor transforms to just functional

* fix import

* fix test

* fix prototype features and functional transforms after review (#5377)

* fix prototype functional transforms after review

* address features review

* make mypy more strict on prototype features

* make mypy more strict for prototype transforms

* fix annotation

* fix kernel tests

* add automatic feature type dispatch to functional transforms (#5323)

* add auto dispatch

* fix missing arguments error message

* remove pil kernel for erase

* automate feature specific parameter detection

* fix typos

* cleanup dispatcher call

* remove __torch_function__ from transform dispatch

* remove auto-generation

* revert unrelated changes

* remove implements decorator

* change register parameter order

* change order of transforms for readability

* add documentation for __torch_function__

* fix mypy

* inline check for support

* refactor kernel registering process

* refactor dispatch to be a regular decorator

* split kernels and dispatchers

* remove sentinels

* replace pass with ...

* appease mypy

* make single kernel dispatchers more concise

* make dispatcher signatures more generic

* make kernel checking more strict

* revert doc changes

* address Franciscos comments

* remove inplace

* rename kernel test module

* fix inplace

* remove special casing for pil and vanilla tensors

* address comments

* update docs

* cleanup features / transforms feature branch (#5406)

* mark candidates for removal

* align signature of resize_bounding_box with corresponding image kernel

* fix documentation of Feature

* remove interpolation mode and antialias option from resize_segmentation_mask

* remove or privatize functionality in features / datasets / transforms

* Add PCAM dataset to prototype area

* use BytesIO instead of writing file to disk

* Apply suggestions from code review
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* remove noqa

* typos

* Use _Resource namedtuple

* Add h5py to unittest_prototype job

* use .item() on target

* Forgot to call regenerate.py
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* use fixture for dataset root in tests

* fix home dir generation

* remove caching
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* add SVHN prototype dataset

* add test

* refactor prototype dataset tests to use public API for loading

* add explanation

* use loop alternative

* refactor prototype datasets tests

* skip tests with insufficient third party dependencies

* cleanup

* add tests for SBD prototype dataset

* add tests for SEMEION prototype dataset

* add tests for VOC prototype dataset

* add tests for CelebA prototype dataset

* add tests for DTD prototype dataset

* add tests for FER2013 prototype dataset

* add tests for CLEVR prototype dataset

* add tests for oxford-iiit-pet prototype dataset

* enforce tests for new datasets

* add missing archive generation for oxford-iiit-pet tests

* add tests for CUB200 prototype datasets

* fix split generation

* add capability to mark parametrization and xfail cub200 traverse tests

* refactor prototype datasets tests

* skip tests with insufficient third party dependencies

* add download functionality to prototype datasets

* fix annotation

* fix test

* remove iopath

* add comments

* improve COCO prototype

* test 2017 annotations

* add option to include captions

* fix categories and add tests

* cleanup

* add correct image size to bounding boxes

* fix annotation collation

* appease mypy

* add benchmark

* always use image as reference

* another refactor

* add support for segmentations

* add support for segmentations

* fix CI dependencies

* return features instead of vanilla tensors from prototype datasets

* fix tests

* remove inplace

* add explanation for __init_subclass__

* fix label for test split

* relax test

* remove pixels

* add test split for imagenet

* add infinite buffer size to shuffler

* add prototype features

* add some JIT tests

* refactor input data handling

* refactor tests

* cleanup tests

* add BoundingBox feature

* mypy

* xfail torchscript tests for now

* cleanup

* fix imports

* add tests for builtin prototype datasets

* fix caltech101

* fix emnist

* fix mnist and variants

* add iopath as test requirement

* fix MNIST warning

* fix qmnist data generation

* fix cifar data generation

* add tests for imagenet

* cleanup