GitLab

* Update EMA every X iters.

* Adding AdamW optimizer.

* Adjusting EMA decay scheme.

* Support custom weight decay for Normalization layers.

* Fix identation bug.

* Change EMA adjustment.

* Quality of life changes to faciliate testing

* ufmt format

* Fixing imports.

* Adding FixRes improvement.

* Support EMA in store_model_weights.

* Adding interpolation values.

* Change train_crop_size.

* Add interpolation option.

* Removing hardcoded interpolation and sizes from the scripts.

* Fixing linter.

* Incorporating feedback from code review.

* Updated classification reference script to use torch.cuda.amp

* Assigned scaler to None if amp is False

* Fixed linter errors

* add ufmt as code formatter

* cleanup

* quote ufmt requirement

* split imports into more groups

* regenerate circleci config

* fix CI

* clarify local testing utils section

* use ufmt pre-commit hook

* split relative imports into local category

* Revert "split relative imports into local category"

This reverts commit f2e224cde2008c56c9347c1f69746d39065cdd51.

* pin black and usort dependencies

* fix local test utils detection

* fix ufmt rev

* add reference utils to local category

* fix usort config

* remove custom categories sorting

* Run pre-commit without fixing flake8

* got a double import in merge
Co-authored-by: Nicolas Hug <nicolashug@fb.com>

* initial code

* add SqueezeExcitation

* initial code

* add SqueezeExcitation

* add SqueezeExcitation

* regnet blocks, stems and model definition

* nit

* add fc layer

* use Callable instead of Enum for block, stem and activation

* add regnet_x and regnet_y model build functions, add docs

* remove unused depth

* use BN/activation constructor and ConvBNActivation

* add expected test pkl files

* allow custom activation in SqueezeExcitation

* use ReLU as the default activation

* initial code

* add SqueezeExcitation

* initial code

* add SqueezeExcitation

* add SqueezeExcitation

* regnet blocks, stems and model definition

* nit

* add fc layer

* use Callable instead of Enum for block, stem and activation

* add regnet_x and regnet_y model build functions, add docs

* remove unused depth

* use BN/activation constructor and ConvBNActivation

* reuse SqueezeExcitation from efficientnet

* refactor RegNetParams into BlockParams

* use nn.init, replace np with torch

* update README

* construct model with stem, block, classifier instances

* Revert "construct model with stem, block, classifier instances"

This reverts commit 850f5f3ed01a2a9b36fcbf8405afd6e41d2e58ef.

* remove unused blocks

* support scaled model

* fuse into ConvBNActivation

* make reset_parameters private

* fix type errors

* fix for unit test

* add pretrained weights for 6 variant models, update docs

* Replaced ToTensor with a combination of PILToTensor and ConvertImageDtype

* Pass dtype

* Adding ExponentialLR and LinearLR

* Fix arg type of --lr-warmup-decay

* Adding support of Zero gamma BN and SGD with nesterov.

* Fix --lr-warmup-decay for video_classification.

* Update bn_reinit

* Fix pre-existing bug on num_classes of model

* Remove zero gamma.

* Use fstrings.

* Warmup on Classficiation references.

* Adjust epochs for cosine.

* Warmup on Segmentation references.

* Warmup on Video classification references.

* Adding support of both types of warmup in segmentation.

* Use LinearLR in detection.

* Fix deprecation warning.

* Add RandomMixupCutmix.

* Add test with real data.

* Use dataloader and collate in the test.

* Making RandomMixupCutmix JIT scriptable.

* Move out label_smoothing and try roll instead of flip

* Adding mixup/cutmix in references script.

* Handle one-hot encoded target in accuracy.

* Add support of devices on tests.

* Separate Mixup from Cutmix.

* Add check for floats.

* Adding device on expect value.

* Remove hardcoded weights.

* One-hot only when necessary.

* Fix linter.

* Moving mixup and cutmix to references.

* Final code clean up.

* Update log message.

* Update fstring.

* add pre-commit hooks

* ignore yamls in packaging/*

* add pre-commit to contributing guide lines

* Update CONTRIBUTING.md
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>

* remove some hooks

* fix docstrings

* fix end of files
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>

* Fix training resuming in references/segmentation

* Clarification for training resnext101_32x8d

* Update references/classification/README.md
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>

* Added Exponential Moving Average support to classification reference script

* Addressed review comments

* Updated model argument

* Initial Proposal

* Tensor Save Test + Test Name Fix

* Formatting + removing unused argument

* fix old argument

* fix isnan check error + indexing error with jit

* Fix Flake8 error.

* Fix MyPy error.

* Fix Flake8 error.

* Fix PyTorch JIT error in UnitTests due to type annotation.

* Fixing tests.

* Removing type ignore.

* Adding support of ta_wide in references.

* Move methods in classes.

* Moving new classes to the bottom.

* Specialize to TA to TAwide

* Add missing type

* Fixing lint

* Fix doc

* Fix search space of TrivialAugment.
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>
Co-authored-by: Vasilis Vryniotis <vvryniotis@fb.com>

* Adding randaugment implementation

* Refactoring.

* Adding num_magnitude_bins.

* Adding FIXME.

* Adding label smoothing on classification reference.

* Replace underscore with dash.

* Adding code skeleton

* Adding MBConvConfig.

* Extend SqueezeExcitation to support custom min_value and activation.

* Implement MBConv.

* Replace stochastic_depth with operator.

* Adding the rest of the EfficientNet implementation

* Update torchvision/models/efficientnet.py

* Replacing 1st activation of SE with SiLU.

* Adding efficientnet_b3.

* Replace mobilenetv3 assets with custom.

* Switch to standard sigmoid and reconfiguring BN.

* Reconfiguration of efficientnet.

* Add repr

* Add weights.

* Update weights.

* Adding B5-B7 weights.

* Update docs and hubconf.

* Fix doc link.

* Fix typo on comment.

* Add submitit script, partition param and parser on its own method.

* Fix method names, handle add_help correctly and refactoring.

* Delete run_with_submitit.py file

* Adding the average_checkpoints() method.

* Adding the store_model_weights() method.

* Adding TODO placeholders.

* More placeholders.

* Add MobileNetV3 small pre-trained weights.

* Remove placeholders.

* Refactoring mobilenetv3 to make code reusable.

* Adding quantizable MobileNetV3 architecture.

* Fix bug on reference script.

* Moving documentation of quantized models in the right place.

* Update documentation.

* Workaround for loading correct weights of quant model.

* Update weight URL and readme.

* Adding eval.

* Adding presets in the classification reference scripts.

* Adding presets in the object detection reference scripts.

* Adding presets in the segmentation reference scripts.

* Adding presets in the video classification reference scripts.

* Moving flip at the end to align with image classification signature.

* Add MobileNetV3 Architecture in TorchVision (#3182)

* Adding implementation of network architecture

* Adding rmsprop support on the train.py

* Adding auto-augment and random-erase in the training scripts.

* Adding support for reduced tail on MobileNetV3.

* Tagging blocks with comments.

* Adding documentation, pre-trained model URL and a minor refactoring.

* Handling better untrained supported models.

* remove unused imports after manual review

* Update torchvision/datasets/voc.py
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

* remove two more instances
Co-authored-by: Ben Weinstein <benweinstein@Bens-MacBook-Pro.local>
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

Summary:

We've made two recent changes to QAT in PyTorch core:
1. add support for SyncBatchNorm
2. make eager mode QAT prepare scripts respect device affinity

This PR updates the torchvision QAT reference script to take
advantage of both of these.  This should be landed after
https://github.com/pytorch/pytorch/pull/39337 (the last PT
fix) to avoid compatibility issues.

Test Plan:

```
python -m torch.distributed.launch
  --nproc_per_node 8
  --use_env
  references/classification/train_quantization.py
  --data-path {imagenet1k_subset}
  --output-dir {tmp}
  --sync-bn
```

Reviewers:

Subscribers:

Tasks:

Tags:

Summary:

Redo of https://github.com/pytorch/vision/pull/2191

Makes the classification QAT tutorial not crash when used
with DDP. There were two issues:

1. the model was moved to GPU before the observers were added, and they
are created on CPU. In the context of this repo, the fix is to finalize
the model before moving to GPU. We can potentially follow up with a
better error message in the future, in a separate PR.
2. the QAT conversion was running on the DDP'ed model, which had various
problems. The fix is to unwrap the model from DDP before cloning it for
evaluation.

There is still work to do on verifying that BN is working correctly in
QAT + DDP, but saving that for a separate PR.

Test Plan:

```
python -m torch.distributed.launch --use_env references/classification/train_quantization.py --data-path {path_to_imagenet_1k} --output_dir {output_dir}
```

Reviewers:

Subscribers:

Tasks:

Tags:

* remove sys.version_info == 2

* remove sys.version_info < 3

* remove from __future__ imports

* add default parameters to README

* fix vgg_*_bn

usage and performance are from the vision0.5 release notes.