GitLab

Co-authored-by: Nicolas Hug <nicolashug@fb.com>

* add ufmt as code formatter

* cleanup

* quote ufmt requirement

* split imports into more groups

* regenerate circleci config

* fix CI

* clarify local testing utils section

* use ufmt pre-commit hook

* split relative imports into local category

* Revert "split relative imports into local category"

This reverts commit f2e224cde2008c56c9347c1f69746d39065cdd51.

* pin black and usort dependencies

* fix local test utils detection

* fix ufmt rev

* add reference utils to local category

* fix usort config

* remove custom categories sorting

* Run pre-commit without fixing flake8

* got a double import in merge
Co-authored-by: Nicolas Hug <nicolashug@fb.com>

Reviewed By: fmassa

Differential Revision: D28537606

fbshipit-source-id: beaf15f4e38a7d986ef26587383aa2aee3e48952

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/3823

Uploaded FashionMNIST dataset to [manifold](https://www.internalfb.com/intern/network/manifold/?bucket=torchvision&path=tree%2Fdatasets) bucket `torchvision`. Any new dataset that needs to be added could be uploaded under `tree/datasets/<dataset_name>`.

Reviewed By: datumbox

Differential Revision: D28358470

fbshipit-source-id: 6f2282d3f1ce4b1416e962de8fb132896d4b2d76

* fix check for exceeded quota on Google Drive

* safeguard quota exceed check

* generalize extract_archive

* [test] re-enable extraction tests on windows

* add tests for detect_file_type

* add error messages to detect_file_type

* Revert "[test] re-enable extraction tests on windows"

This reverts commit 7fafebb0f6b4c49bd72c4b5e0a0b4b8c432bce57.

* add utility functions for better mock call checking

* add tests for decompress

* simplify logic by using pathlib

* lint

* Apply suggestions from code review
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

* make decompress private

* remove unnecessary checks

* add error message

* fix mocking

* add remaining tests

* lint
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

* use head request for redirects

* remove requests dependency

* add custom user agent for download_url

* fix progress bar

* lint

* [test] use repo instead of nightly for download tests

* Check if the file is located on Google Drive

download_file() detects if the provided URL contains a link to Google
Drive and passes the request through the download_file_from_google_drive()
if it is the case.

* remove lazy re import

* add guard clause

* use urlparse

* add tests

* Fixing lint.

* make id matching more safe

* Fixing type check.

* Remove AnyStr.
Co-authored-by: Philip Meier <github.pmeier@posteo.de>
Co-authored-by: Vasilis Vryniotis <vvryniotis@fb.com>

* Make download_url() follow redirects

Fix bug related to the incorrect processing of redirects.
Follow the redirect chain until the destination is reached or
the number of redirects exceeds the max allowed value (by default 10).

* Parametrize value of max allowed redirect number

Make max number of hops a function argument and assign its default value to 10

* Propagate the max number of hops to download_url()

Add the maximum number of redirect hops parameter to download_url()

* check file existence before redirect

* remove print

* remove recursion

* add tests

* Reducing max_redirect_hops
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>
Co-authored-by: Philip Meier <github.pmeier@posteo.de>
Co-authored-by: Vasilis Vryniotis <vvryniotis@fb.com>

* remove unused imports after manual review

* Update torchvision/datasets/voc.py
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

* remove two more instances
Co-authored-by: Ben Weinstein <benweinstein@Bens-MacBook-Pro.local>
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

* enable mypy for torchvision.datasets and ignore existing errors

* imagenet

* utils

* lint

Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

Refactoring to use contexts managers, list comprehensions when more idiomatic, and minor renaming to help reader clarity (#2335)

* Refactoring to use contexts managers, list comprehensions when more idiomatic, and minor renaming to help reader clarity.

* Fix flake8 warning in video_utils.py

* remove six from python code

* remove six from setup.py

* remove six from tests

* remove six from references

* remove six from packaging

* revert str to torch._six._string_classes

* revert str to torch._six._string_classes

* tgz updates

* tgz updates

* tgz updates

* Removed unnecessary class variables.

* The integrity of dataset files is now being checked right after the download finished. Thus making sure that a corrupt file is not being extracted. In case of corruption we throw a RuntimeError.

* Added missing md5 hashes to MNIST, FashionMNIST, KMNIST, EMNIST and QMNIST datasets.

* Removed printing of error message when integrity check failed.
Reformulated error message.

* Reformatted code to be lint conform.

* Fixed formatting in utils.py

* Fix nightly builds

* Fix lint

* Disable video_reader tests on Travis

* Disable one more test that segfaults on Travis

* Fix typo in expression

* add tar.xz archive handler

* update unittest for tar.xz archive

* remove .tar.xz unittest

* add separate .tar.xz unittest

* update PY2 compatibility

Following a similar line of inquiry to pytorch/audio#217

* introduced function to verify str arguments

* flake8

* added FIXME to VOC

* Fixed error message

* added test for verify_str_arg

* cleanup todos

* added option for custom error message

* fix VOC

* fixed Caltech

* Raise error during downloading

* Fix py2 error and lint

* added fake data

* fixed fake data

* renamed extract and download methods and added functionality

* added raw fake data

* refactored imagenet and added test

* flake8

* added fake devkit and mocked download_url

* reversed uncommenting

* added mock to CI

* fixed tests for imagefolder

* flake8

* WIP

* WIP: minor improvements

* Add tests

* Fix typo

* Use download_and_extract on caltech, cifar and omniglot

* Add a print message during extraction

* Remove EMNIST from test

* fixed check integrity

* stylistic changes

* flake8

* Add Caltech101 and Caltech256

* Add information about default for target_type

* Fix docs

* Add function to download from Google Drive

* Add CelebA dataset

* Only import pandas when needed

* Addressing comments

* Remove trailing whitespace

* Replace torch.LongTensor with torch.as_tensor

* Fixes for PyTorch version of tqdm

* Flake

* Flake fix

* Add SBU Captioned Photo Dataset

* Add SBU to the dataset docs

* cast mnist targer to int

* fix unused variables

* fix syntax in tests

* remove # noqa; rename l variable to line_split; use except OSError

* add W504

* add W504

* add W503,W504 tox.ini

* MNIST and FashionMNIST now have their own 'raw' and 'processed' folders

* mkdir exist_ok