GitLab

* use head request for redirects

* remove requests dependency

* add custom user agent for download_url

* fix progress bar

* lint

* [test] use repo instead of nightly for download tests

* Check if the file is located on Google Drive

download_file() detects if the provided URL contains a link to Google
Drive and passes the request through the download_file_from_google_drive()
if it is the case.

* remove lazy re import

* add guard clause

* use urlparse

* add tests

* Fixing lint.

* make id matching more safe

* Fixing type check.

* Remove AnyStr.
Co-authored-by: Philip Meier <github.pmeier@posteo.de>
Co-authored-by: Vasilis Vryniotis <vvryniotis@fb.com>

* Make download_url() follow redirects

Fix bug related to the incorrect processing of redirects.
Follow the redirect chain until the destination is reached or
the number of redirects exceeds the max allowed value (by default 10).

* Parametrize value of max allowed redirect number

Make max number of hops a function argument and assign its default value to 10

* Propagate the max number of hops to download_url()

Add the maximum number of redirect hops parameter to download_url()

* check file existence before redirect

* remove print

* remove recursion

* add tests

* Reducing max_redirect_hops
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>
Co-authored-by: Philip Meier <github.pmeier@posteo.de>
Co-authored-by: Vasilis Vryniotis <vvryniotis@fb.com>

* remove unused imports after manual review

* Update torchvision/datasets/voc.py
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

* remove two more instances
Co-authored-by: Ben Weinstein <benweinstein@Bens-MacBook-Pro.local>
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

* enable mypy for torchvision.datasets and ignore existing errors

* imagenet

* utils

* lint

Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

Refactoring to use contexts managers, list comprehensions when more idiomatic, and minor renaming to help reader clarity (#2335)

* Refactoring to use contexts managers, list comprehensions when more idiomatic, and minor renaming to help reader clarity.

* Fix flake8 warning in video_utils.py

* remove six from python code

* remove six from setup.py

* remove six from tests

* remove six from references

* remove six from packaging

* revert str to torch._six._string_classes

* revert str to torch._six._string_classes

* tgz updates

* tgz updates

* tgz updates

* Removed unnecessary class variables.

* The integrity of dataset files is now being checked right after the download finished. Thus making sure that a corrupt file is not being extracted. In case of corruption we throw a RuntimeError.

* Added missing md5 hashes to MNIST, FashionMNIST, KMNIST, EMNIST and QMNIST datasets.

* Removed printing of error message when integrity check failed.
Reformulated error message.

* Reformatted code to be lint conform.

* Fixed formatting in utils.py

* Fix nightly builds

* Fix lint

* Disable video_reader tests on Travis

* Disable one more test that segfaults on Travis

* Fix typo in expression

* add tar.xz archive handler

* update unittest for tar.xz archive

* remove .tar.xz unittest

* add separate .tar.xz unittest

* update PY2 compatibility

Following a similar line of inquiry to pytorch/audio#217

* introduced function to verify str arguments

* flake8

* added FIXME to VOC

* Fixed error message

* added test for verify_str_arg

* cleanup todos

* added option for custom error message

* fix VOC

* fixed Caltech

* Raise error during downloading

* Fix py2 error and lint

* added fake data

* fixed fake data

* renamed extract and download methods and added functionality

* added raw fake data

* refactored imagenet and added test

* flake8

* added fake devkit and mocked download_url

* reversed uncommenting

* added mock to CI

* fixed tests for imagefolder

* flake8

* WIP

* WIP: minor improvements

* Add tests

* Fix typo

* Use download_and_extract on caltech, cifar and omniglot

* Add a print message during extraction

* Remove EMNIST from test

* fixed check integrity

* stylistic changes

* flake8

* Add Caltech101 and Caltech256

* Add information about default for target_type

* Fix docs

* Add function to download from Google Drive

* Add CelebA dataset

* Only import pandas when needed

* Addressing comments

* Remove trailing whitespace

* Replace torch.LongTensor with torch.as_tensor

* Fixes for PyTorch version of tqdm

* Flake

* Flake fix

* Add SBU Captioned Photo Dataset

* Add SBU to the dataset docs

* cast mnist targer to int

* fix unused variables

* fix syntax in tests

* remove # noqa; rename l variable to line_split; use except OSError

* add W504

* add W504

* add W503,W504 tox.ini

* MNIST and FashionMNIST now have their own 'raw' and 'processed' folders

* mkdir exist_ok

- Fix broken update calculation
- Make progress bar use the neat `unit_scale` feature
  of tqdm

* Added Progress Bar to the dataset downloading utility

* Updated progressbar to update by blcoks

* Added progressbar2 to the requirements

* fixed style issues

* Shifted from progressbar to tqdm

* Add basic Omniglot dataset loader

* Remove unused import

* Add Omniglot random pair to sample pair of characters

* Precompute random set of pairs, deterministic after object instantiation

* Export OmniglotRandomPair via the datasets module interfact

* Fix naming convention, use sum instead of reduce

* Fix downloading to not download everything, fix Python2 syntax

* Fix end line lint

* Add random_seed, syntax fixes

* Remove randomized pair, take up as a separate generic wrapper

* Fix master conflict

According to the comment, this should be 1MB which is 1024*1024 bytes.