GitLab

Summary:
This diff enables ImageNet to be usable from within torchvision.

It implements a different codepath for some families of datasets that can be sharded into multiple pieces, and uses it if within fbcode.

For now all previously supported datasets are also supported (MNIST, CIFAR and FashionMNIST), in addition to ImageNet.

Pull Request resolved: https://github.com/pytorch/vision/pull/4776

Reviewed By: datumbox

Differential Revision: D31929684

fbshipit-source-id: 58f9fe2ed6ce731a6b43a38be912983247eff562
(cherry picked from commit b43fead8ea5164e37637ade0e80477e94f993365)

* add ufmt as code formatter

* cleanup

* quote ufmt requirement

* split imports into more groups

* regenerate circleci config

* fix CI

* clarify local testing utils section

* use ufmt pre-commit hook

* split relative imports into local category

* Revert "split relative imports into local category"

This reverts commit f2e224cde2008c56c9347c1f69746d39065cdd51.

* pin black and usort dependencies

* fix local test utils detection

* fix ufmt rev

* add reference utils to local category

* fix usort config

* remove custom categories sorting

* Run pre-commit without fixing flake8

* got a double import in merge
Co-authored-by: Nicolas Hug <nicolashug@fb.com>