GitLab

* Broken down PR(#6269). Added an additional dataset

* Removed some types. Store None instead of "". Merged test util functions.

* minor mypy fixes. minor doc fixes

* reformated docstring

* Added additional line-skips

* upgrade usort to

* Also update black

* Actually use 1.0.2

* Apply pre-commit
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>

Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>

* Adding multiweight support for shufflenetv2 prototype models

* Revert "Adding multiweight support for shufflenetv2 prototype models"

This reverts commit 31fadbee7d1a65cd73ae43dfd4ac6e97e7ca7b01.

* Adding multiweight support for shufflenetv2 prototype models

* Revert "Adding multiweight support for shufflenetv2 prototype models"

This reverts commit 4e3d900f796c1e3e667312087e77956ca4a4c017.

* Add RenderedSST2 dataset

* Address PR comments

* Fix bug in dataset verification
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>

* [WIP]
*added stanford_cars

* [WIP]
added stanfordCars to docs

* [WIP]
minor edits

* [WIP]
minor edits

* edited StanfordCars class

* Adding Testcase for stanford cars

* Added Testcase for stanford cars

* Added Testcase for stanford cars

* minor edit

* made changes as per the suggestions

* fixed typo in naming stanford_cars.py

* cars_meta.mat file will be created in test

* Some cleanups

* Sigh

* don't convert to strings
Co-authored-by: Nicolas Hug <nicolashug@fb.com>

* feat: Added EuroSAT dataset

* test: Added unittest

* docs: Improved comments

* docs: Updated the documentation

* docs: Removed unnecessary comments

* fix: Fixed class implementation

* test: Fixed unittest

* fix: Fixed magic method len

* test: Fixed unittest

* refactor: Refactored EuroSAT

* refactor: Applied modifications

* Apply suggestions from code review

* refactor: Applied request changes

* refactor: Made var explicit

* fix: Fixed attribute initialization order

* refactor: Removed name mapping
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* Add support for PCAM dataset

* mypy

* Apply suggestions from code review
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* Remove classes and class_to_idx attributes

* Use _decompress
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* add fvgc_aircraft dataset

* add docstring & remove useless import

* resolve lint issue

* address comments

* adding more annotation level

* nit

* address comments

* Apply suggestions from code review

* unify format

* remove useless line
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* Add Flowers102 datasets

* Fix initialization of images and labels

* Fix _check_exists in Flowers102

* Add Flowers102 to datasets and docs

* Add Flowers102TestCase to unittest

* Fixing Python type statically

* Shuffle the fake labels

* Update test/test_datasets.py
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* Apply the suggestions by pmeier

* Use check_integrity to check file existence

* Save the labels to base_folder

* Minor fixes

* Using a loop makes this more concise without reducing readability
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* Using a loop makes this more concise without reducing readability
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* Remove self.labels and self.label_to_index attributes

* minor simplification

* Check the exitence of image folder

* Revert the check

* Check the existence of image folder

* valid -> val

* keep some stuff private

* minor doc arrangements

* remove default FEATURE_TYPES

* Simplify the datasets existence

* check if the image folder exists
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* isdir -> is_dir
Co-authored-by: Philip Meier <github.pmeier@posteo.de>
Co-authored-by: Nicolas Hug <nicolashug@fb.com>

* Add Country211 dataset

To addresses issue #5108.

* Add Country211 dataset

To addresses issue #5108.

* Update country211.py

* Update country211.py

* Code review reflected

Reflect code review

* Update test_datasets.py

* Update with review

Update with review

* inherit from ImageFolder

* Update test/test_datasets.py

* Docstring + minor test update
Co-authored-by: Philip Meier <github.pmeier@posteo.de>
Co-authored-by: Nicolas Hug <nicolashug@fb.com>

* dataset class added

* fix code format

* fixed requested changes

* fixed issues in sun397

* Update torchvision/datasets/sun397.py
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* add prototype dataset for oxford-iiit-pet

* add old-style dataset

* add tests

* fix mypy

* fix test

* remove properties and use pathlib

* target_type to target_types

* move target annotation

* add docstring

* fix test

* add prototype dataset

* add old-style dataset

* appease mypy

* simplify prototype scenes

* Update torchvision/datasets/clevr.py
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>

* Added GTSRB dataset

* Added unittest for GTSRB dataset

* Apply suggestions from code review

* More changes from code review

* readd accidental removed line
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* add prototype dataset

* add old style dataset

* Apply suggestions from code review
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>

* refactor integrity check
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>

* add DTD as prototype dataset

* add old style dataset

* add test for old dataset

* fix tests for windows

* add dataset to docs

* remove properties and use pathlib

* Apply suggestions from code review
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>

* fold -> partition
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>

* Adding multiweight support for shufflenetv2 prototype models

* Revert "Adding multiweight support for shufflenetv2 prototype models"

This reverts commit 31fadbee7d1a65cd73ae43dfd4ac6e97e7ca7b01.

* Adding multiweight support for shufflenetv2 prototype models

* Revert "Adding multiweight support for shufflenetv2 prototype models"

This reverts commit 4e3d900f796c1e3e667312087e77956ca4a4c017.

* Add Food101 Dataset

Addresses #5108.
cc @pmeier @NicolasHug

* Remove unecessary Path contructor calls

* Remove unecessary Path contructor callsi and fix types

* Fix tests

* Address PR comments from @pmeier

* Fix bug in tests and in food101 dataset

* Fix bug in tests and in food101 dataset

* Update torchvision/datasets/food101.py
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* add ufmt as code formatter

* cleanup

* quote ufmt requirement

* split imports into more groups

* regenerate circleci config

* fix CI

* clarify local testing utils section

* use ufmt pre-commit hook

* split relative imports into local category

* Revert "split relative imports into local category"

This reverts commit f2e224cde2008c56c9347c1f69746d39065cdd51.

* pin black and usort dependencies

* fix local test utils detection

* fix ufmt rev

* add reference utils to local category

* fix usort config

* remove custom categories sorting

* Run pre-commit without fixing flake8

* got a double import in merge
Co-authored-by: Nicolas Hug <nicolashug@fb.com>

* Added LFW Dataset

* Added dataset to list in __init__.py

* Updated lfw.py
* Created a common superclass for people and pairs type datatsets
* corrected the .download() method

* Added docstrings and updated datasets.rst

* Wrote tests for LFWPeople and LFWPairs

* Resolved mypy error: Need type annotation for "data"

* Updated inject_fake_data method for LFWPeople

* Updated tests for LFW

* Updated LFW tests and minor changes in lfw.py

* Updated LFW
* Added functionality for 10-fold validation view
* Optimized the code so to replace repeated lines by method in super
  class
* Updated LFWPeople to get classes from all lfw-names.txt rather than
  just the classes fron trainset

* Updated lfw.py and tests
* Updated inject_fake_data method to create 10fold fake data
* Minor changes in docstring and extra_repr

* resolved py lint errors

* Added checksums for annotation files

* Minor changes in test

* Updated docstrings, defaults and minor changes in test

* Removed 'os.path.exists' check
Co-authored-by: ABD-01 <myac931@gmai.com>
Co-authored-by: Philip Meier <github.pmeier@posteo.de>
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

* Add iNaturalist dataset

* Add download support

* address comments
Co-authored-by: dgenzel <dgenzel@fb.com>
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

* Initial commit

* pmeiers comments
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* pmeiers changes
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* pmeiers comments
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* replace pandas with system library to avoid crashes

* Lint

* Lint

* fixing unittest

* Minor comments removal

* pmeier comments
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* remove asserts

* address pmeier formatting changes

* address pmeier changes
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* pmeier changes
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* rename n_classes to num_classes

* formatting changes

* doc change to add ".mp4" to backported class

* formatting to correct line length

* adding **kwargs to Kinetics400 class

* remove urlib request and download the file directly

* annotations and files can be already downloaded

* test fix

* add download tests for Kinetics

* users now dont need to provide full path within the root for new Kinetics dataset

* linter

* Update test/test_datasets_download.py

* Update torchvision/datasets/kinetics.py
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* revert whitespace (3680#discussion_r626382842)

* addressing annotation_path parameter which is unnecessary

* Update torchvision/datasets/kinetics.py
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* Update torchvision/datasets/kinetics.py
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* kwargs update
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* expose num_download_workers as public

* swap os.isfile with check_integrity

* nit on private things

* special case if there are no default arguments

* revert changes to kinetics400 test case for BC

* add split_folder changes and support for legacy format

* pmeiers suggestions
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* pmeiers suggestions - root comment

* pmeiers comments - annotation attribute remmoved

* pmeiers suggestion
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* pmeiers suggestion
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* pmeiers suggestion
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* pmeiers suggestion
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* Update torchvision/datasets/kinetics.py
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* Update torchvision/datasets/kinetics.py
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* Update torchvision/datasets/kinetics.py
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* Update torchvision/datasets/kinetics.py
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* Update torchvision/datasets/kinetics.py
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* Update torchvision/datasets/kinetics.py
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* minor debugging

* nit picks

* only include public kwargs into defaults

* add _use_legacy_structure in favour of **kwargs

* add type hints for Kinetics400

* flake8

* flake8

* flake8

* rename to make thigs clearer

* permuting the output
Co-authored-by: Philip Meier <github.pmeier@posteo.de>
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

* Added KITTI dataset

* Addressed review comments

* Changed type of target to List[Dict] and corrected the data types of the returned values.

* Updated unit test to rely on ImageDatasetTestCase

* Added kitti to dataset documentation

* Cleaned up test and some minor changes

* Made data_url a string instead of a list

* Removed unnecessary try and print
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

* initial commit of widerface dataset

* comment out old code

* improve parsing of annotation files

* code cleanup and fix docstring comments

* speed up check for quota exceeded

* cleanup print statements

* reformat code and remove print statements

* minor code cleanup and reformatting

* add more comments

* reuse variable

* reverse formatting changes

* fix flake8 errors

* add type annotations

* fix mypy errors

* add a base_folder to root directory

* some formatting fixes

* GDrive threshold does not throw 403 error

* testing new download logic

* cleanup logic for download and integrity check

* use a better variable name

* format fix

* reorder list in docstring

* initial widerface unit test - fails on MD5 check

* use list of dictionaries to store dataset

* fix docstring formatting

* remove unnecessary error checking

* fix type checker error

* revert typo fix

* rename var constants, use file context manager, verify str args

* fix flake8 error

* fix checking target_type argument values

* create uncompressed dataset folders

* cleanup unit tests for widerface

* use correct os function

* add more info to docstring

* disable unittests for windows

* fix _check_integrity logic

* update docstring

* remove citation

* remove target_type option

* fix formatting issue
Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* remove comment and add more info to docstring

* update type annotations

* restart CI jobs
Co-authored-by: Joshua Bradley <jgbrad3@evoforge.org>
Co-authored-by: Philip Meier <github.pmeier@posteo.de>
Co-authored-by: vfdev <vfdev.5@gmail.com>

* initial draft

* [dirty] progress

* remove inheritance from ImageFolder

* add tests

* lint

* fix type hints

* align getitem with other datasets

* remove unused import

* add docstring

* guard existing image folders from overwrite

* add missing entry in docstring

* make fixpath more legible

* add Places365 to docs

Also add docs

* Add HMDB51 and UCF101

* Remove debug code

* Add VideoClips and Kinetics dataset

* Lint + add back missing line

* Adds ClipSampler following Bruno comment

* Change name following Bruno's suggestion

* Enable specifying a target framerate

* Fix test_io for new interface

* Add comment mentioning drop_last behavior

* Make compute_clips more robust

* Flake8

* Fix for Python2

* Added general reader for sn3 tensors in "pascalvincent" format

* Added class QMNIST into mnist.py

* QMNIST dataset: make some pt files smaller

* Change request from fmassa.

* read_sn3_pascalvincent_tensor: cse

* read_sn3_pascalvincent_tensor: check file size (when strict!=False)

* Fix lint

* More lint

* Add documentation and expose QMNIST to dataset namespace

* add USPS dataset

* minor fixes

* Improvements to the USPS dataset

Add it to the documentation, expose it to torchvision.datasets
and inherit from VisionDataset

* Added Semantic Boundaries Dataset

* Updated docs and datasets importing module

* Improved docs

* initial commit

* fixed Python2 issue

* fixed naming incorrectness and Python2 compability

* fixed preparation of train folder

* removed detection dataset

* added docstring and repr

* moved import of scipy to make the import of torchvision independent of it

* improved conversion from class string to index

* removed support for other years than 2012

* removed accidentally added file

* moved emptying of split folder to avoid accidental deletion

* removed deletion of the images

* removed error conversion for Python2

* Aligned class indices with the indices identified by ImageFolder class

* Add Caltech101 and Caltech256

* Add information about default for target_type

* Fix docs

* Add function to download from Google Drive

* Add CelebA dataset

* Only import pandas when needed

* Addressing comments

* Remove trailing whitespace

* Replace torch.LongTensor with torch.as_tensor

* Add KMNIST

* Add documentation

* Add KMNIST to module's __all__ list

* Add Cityscapes Dataset

* Rename 'label' to 'semantic' to make the meaning more clear

* Add support for gtCoarse target set