GitLab

Refactoring to use contexts managers, list comprehensions when more idiomatic, and minor renaming to help reader clarity (#2335)

* Refactoring to use contexts managers, list comprehensions when more idiomatic, and minor renaming to help reader clarity.

* Fix flake8 warning in video_utils.py

* remove six from python code

* remove six from setup.py

* remove six from tests

* remove six from references

* remove six from packaging

* revert str to torch._six._string_classes

* revert str to torch._six._string_classes

* tgz updates

* tgz updates

* tgz updates

* Removed unnecessary class variables.

* The integrity of dataset files is now being checked right after the download finished. Thus making sure that a corrupt file is not being extracted. In case of corruption we throw a RuntimeError.

* Added missing md5 hashes to MNIST, FashionMNIST, KMNIST, EMNIST and QMNIST datasets.

* Removed printing of error message when integrity check failed.
Reformulated error message.

* Reformatted code to be lint conform.

* Fixed formatting in utils.py

* Fix nightly builds

* Fix lint

* Disable video_reader tests on Travis

* Disable one more test that segfaults on Travis

* Fix typo in expression

* add tar.xz archive handler

* update unittest for tar.xz archive

* remove .tar.xz unittest

* add separate .tar.xz unittest

* update PY2 compatibility

Following a similar line of inquiry to pytorch/audio#217

* introduced function to verify str arguments

* flake8

* added FIXME to VOC

* Fixed error message

* added test for verify_str_arg

* cleanup todos

* added option for custom error message

* fix VOC

* fixed Caltech

* Raise error during downloading

* Fix py2 error and lint

* added fake data

* fixed fake data

* renamed extract and download methods and added functionality

* added raw fake data

* refactored imagenet and added test

* flake8

* added fake devkit and mocked download_url

* reversed uncommenting

* added mock to CI

* fixed tests for imagefolder

* flake8

* WIP

* WIP: minor improvements

* Add tests

* Fix typo

* Use download_and_extract on caltech, cifar and omniglot

* Add a print message during extraction

* Remove EMNIST from test

* fixed check integrity

* stylistic changes

* flake8

* Add Caltech101 and Caltech256

* Add information about default for target_type

* Fix docs

* Add function to download from Google Drive

* Add CelebA dataset

* Only import pandas when needed

* Addressing comments

* Remove trailing whitespace

* Replace torch.LongTensor with torch.as_tensor

* Fixes for PyTorch version of tqdm

* Flake

* Flake fix

* Add SBU Captioned Photo Dataset

* Add SBU to the dataset docs

* cast mnist targer to int

* fix unused variables

* fix syntax in tests

* remove # noqa; rename l variable to line_split; use except OSError

* add W504

* add W504

* add W503,W504 tox.ini

* MNIST and FashionMNIST now have their own 'raw' and 'processed' folders

* mkdir exist_ok

- Fix broken update calculation
- Make progress bar use the neat `unit_scale` feature
  of tqdm

* Added Progress Bar to the dataset downloading utility

* Updated progressbar to update by blcoks

* Added progressbar2 to the requirements

* fixed style issues

* Shifted from progressbar to tqdm

* Add basic Omniglot dataset loader

* Remove unused import

* Add Omniglot random pair to sample pair of characters

* Precompute random set of pairs, deterministic after object instantiation

* Export OmniglotRandomPair via the datasets module interfact

* Fix naming convention, use sum instead of reduce

* Fix downloading to not download everything, fix Python2 syntax

* Fix end line lint

* Add random_seed, syntax fixes

* Remove randomized pair, take up as a separate generic wrapper

* Fix master conflict

According to the comment, this should be 1MB which is 1024*1024 bytes.