GitLab

Summary:
Current implementation of d2go lightning default task fails when running a model training with EMA.
The error is :
```
RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss.
```
The error is due the fact the d2go lightning task create a copy of the ema model for evaluation that does not included in the training, which raise the error that there are unused params.
This is solved by moving the copy creation to after training and to when evaluation starts.

Reviewed By: kazhang

Differential Revision: D33442690

fbshipit-source-id: e9e469e33811de0b4171a64293cc16a8157af08c

Summary:
### New commit log messages
  b64dea9dc Rename `DDPPlugin` to `DDPStrategy` (#11142)

Reviewed By: jjenniferdai

Differential Revision: D33259306

fbshipit-source-id: b4608c6b96b4a7977eaa4ed3f03c4b824882aef0

Summary: Try LSJ with Faster RCNN with FBNet backbone

Reviewed By: newstzpz

Differential Revision: D32054932

fbshipit-source-id: 4fdb30e7b1258d6f167f2c2fd331209aad1b599a

Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/152

Reviewed By: zhanghang1989

Differential Revision: D31591900

fbshipit-source-id: 6ee8124419d535caf03532eda4f729e707b6dda7

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/155

- remove tracing, which shouldn't affect anything.
- create the model in cpu mode, since it might have issue casting model.

Reviewed By: zhanghang1989

Differential Revision: D33357269

fbshipit-source-id: 27a0330ebb12b993744dee47151c3056cd584ccf

Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/154

Reviewed By: zhanghang1989

Differential Revision: D33352204

fbshipit-source-id: e1a9ac6eb2574dfe6931435275e27c9508f66352

Summary: DDPPlugin has been renamed to DDPStrategy (as part of https://github.com/PyTorchLightning/pytorch-lightning/issues/10549), causing oss CI to fail. Simply skipping the import to unblock CI since DDP feature is not used in test.

Reviewed By: kazhang

Differential Revision: D33351636

fbshipit-source-id: 7a1881c8cd48d9ff17edd41137d27a976103fdde

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/153

Fix the error:
'''
ImportError: cannot import name 'Detectron2GoRunner' from 'd2go.runner' (/home/runner/work/d2go/d2go/d2go/runner/__init__.py)
'''

Reviewed By: zhanghang1989

Differential Revision: D33306196

fbshipit-source-id: 6c773260855498ed498bd43f998bc4de891ff9e2

Summary:
1. Add registry for coco injection to allow for easier overriding of cococ injections
2. Coco loading currently is limited to certain keys. Adding option to allow for copying certain keys from the outputs.

Reviewed By: zhanghang1989

Differential Revision: D33132517

fbshipit-source-id: 57ac4994a66f9c75457cada7e85fb15da4818f3e

Summary: Dataset size check happened at https://fburl.com/code/ey18d081 . Adding this assertion to break and detect errors earlier.

Differential Revision: D33108979

fbshipit-source-id: c931a2167fce8ec0ce764365778d879280bd5af4

Summary:
Currently the lightning task rely on the default runner for the vis wrapper logic. This does not allow to overload the get_data_loader_vis_wrapper is subclasses of the lightning task class.
This diff fixes this issue and properly take the vis wrapper given by the overloaded get_data_loader_vis_wrapper functions in the runner.

Reviewed By: zhanghang1989

Differential Revision: D33190410

fbshipit-source-id: 48cb3a8fa4b11df41d025d115d21002991549ced

Summary:
With this diff, we add support for Pseudo-GT / machine-generated labels on a D2 (https://github.com/facebookresearch/d2go/commit/7992f91324aee6ae59795063a007c6837e60cdb8)Go dataset.

The idea is to generate pseudo-GT early on a project and on a rolling basis replace these labels with human-GT as annotation progress occurs (which we know is costly).

## how to use

To add pseudo-GT labels for a class (segmentation only), all that needs to be done is add the mask folder (with machine-generated labels) just like you would for human-generated labels **but with the "_pseudo-label" postfix**.

After this, you'd have to register the dataset following the instructions in D32298220.

## example

For example, say you're adding an **eyebrows** class to a dataset located in: `manifold://pai_mobile/tree/datasets/some_dataset/batch1`.

You'd then add your `eyebrows` folder with all your machine-generated .PNGs as `eyebrows_pseudo-label` in `manifold://pai_mobile/tree/datasets/some_dataset/batch1/mask/eyebrows_pseudo-label`.

After this, you'd have to register the dataset following the instructions in D32298220.

Reviewed By: wenliangzhao2018

Differential Revision: D32298221

fbshipit-source-id: 230a862e6be69306fb5c119b778e14e12d1280e0

Summary: will help debugging

Reviewed By: tglik

Differential Revision: D32771358

fbshipit-source-id: 659c0edc79354ca8688b13058f784f653c0cff37

Summary:
This tool mimics how the e2e_workflow initializes the runner+config pair, which is the only way to obtain the dataset information from the catalog.

This is very early and mostly helps debug. In the near future, this tool is meant to help users understand whether their current setup is compatible with the AIM lineage automation D2 (https://github.com/facebookresearch/d2go/commit/7992f91324aee6ae59795063a007c6837e60cdb8)GO provides, ideally giving clear signal on what to change.

Differential Revision: D32497618

fbshipit-source-id: 83f1d6ae1dca48ce37a05c84b460ea4bb7a1fd79

Reviewed By: zhanghang1989

Differential Revision: D32680045

fbshipit-source-id: ad21fd81a496a10d4d0499de83ff2469b4fcbf00

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/146

OSSpytorch package removed caffe2. This causes https://github.com/facebookresearch/d2go/issues/137

This diff only makes d2go importable without caffe2. But export is still broken when caffe2 is not available.

Reviewed By: zhanghang1989

Differential Revision: D32690938

fbshipit-source-id: d345687bd720b4b1376494478f1fa44f4c591ccf

Summary:
Experimental models from Xiaoliang [D31749820]

Pretrained weights:
fbnet_vit_tiny_v3_lepe n/a
fbnet_deit_v0 f298782311

Reviewed By: XiaoliangDai

Differential Revision: D32054949

fbshipit-source-id: 7c2aa0679a545ed814ba1db421408a5f9a59a2c8

Summary: make it an option

Differential Revision: D32601981

fbshipit-source-id: 308a0c49939531d840914aa8e256aae6db463929

Summary: Pull Request resolved: https://github.com/facebookresearch/mobile-vision/pull/56

Reviewed By: ppwwyyxx

Differential Revision: D32576986

fbshipit-source-id: 1b20d1927a36ac80e33b51ff971b54767f647d43

Summary:
for sythetic data, we want to enable having different transforms for different dataloaders.

To do that, we need to be able to construct different kinds of transforms.

This means that using the cfg's hard-coded location isn't convenient - we'd have to edit the cfg during run time and call the build function multiple times

Differential Revision: D32486576

fbshipit-source-id: 767b63c5c787e31a67dbf8710ab9bab84a0651db

Summary:
### New commit log messages
  fa0ed17f8 remove deprecated train_loop (#10482)

Reviewed By: kandluis

Differential Revision: D32454980

fbshipit-source-id: a35237dde06cc9ddac5373b75992ce88a6771c76

Reviewed By: newstzpz

Differential Revision: D32301322

fbshipit-source-id: a9e951b9de600012125b8b94c0c1ace929b491b8

Summary:
fvcore flops calculator throws on this error:

KeyError: 'Only support flattening dictionaries if keys are str.'

Setting flops to some value so it doesn't enter pdb mode.

Reviewed By: stephenyan1231

Differential Revision: D32144492

fbshipit-source-id: 604cd4660cea9ffbfb3f1da35d32e06ccf607a50

Reviewed By: newstzpz

Differential Revision: D31209906

fbshipit-source-id: 0be4e3c1db623e3c1fba8ba4259840d34192a77e

Summary: Extended Pix2Pix to allow for input extra data

Reviewed By: tax313

Differential Revision: D31469054

fbshipit-source-id: 790543f214ea9fa0158e509acb27193916bf17ce

Reviewed By: zertosh

Differential Revision: D32270982

fbshipit-source-id: 8767b469fe5404a882257c0c5209b34ed0c327dc

Summary: code was kept for short term to support loading old training jobs during the period when the default config is polluted; now it should be safe to remove this BC support and dead code

Differential Revision: D32218217

fbshipit-source-id: 3772477653151ccbcb4ae7098b9414853b581ad1

Reviewed By: sstsai-adl

Differential Revision: D32216605

fbshipit-source-id: bebee1edae85e940c7dcc6a64dbe341a2fde36a2

Reviewed By: jreese, ppwwyyxx

Differential Revision: D32191010

fbshipit-source-id: 1e40b7a090be3a0e25b930fb908ec177719fce50

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/134

Updating `all_steps_qat` example config to use learnable QAT method.

And add logic in `GeneralizedRCNNPatch.prepare_for_quant` to call the new `d2go.utils.qat_utils.get_qat_qconfig` to properly support QAT in D2 (https://github.com/facebookresearch/d2go/commit/7992f91324aee6ae59795063a007c6837e60cdb8)Go training workflow

Differential Revision: D32147216

fbshipit-source-id: 32831c6156bc5c0775196ad8edc890a5292d204f

Summary: Allow reading `.npy` format binary masks shaped (H, W,) in addition to `.png` image masks shaped (H, W, C).

Reviewed By: wat3rBro

Differential Revision: D30136542

fbshipit-source-id: 56df5a766ab15b6808a1327815857e5d38eac910

Summary:
In quantization callback, we prepare the model with FX quantization API and only use the prepared model in training.
However, when training in DDP, the parameters in the origin model still require grad, causing unused parameters RuntimeError.
Previously, Lightning trainer train the model with find_unused_param flag, but if user manually disable it, they will get the runtime error.

In this diff, the parameters in the origin model are frozen. We could consider deleting the origin model after preparation to save memory, but we might have to make some assumption on Lightning module structure, for example, `.model` is the origin model, so that we could `delattr(pl_module, "model")`.

Reviewed By: wat3rBro

Differential Revision: D31902368

fbshipit-source-id: 56eabb6b2296278529dd2b94d6aa4c9ec9e9ca6b

Summary: as title

Reviewed By: Cysu

Differential Revision: D31901433

fbshipit-source-id: 1749527c04c392c830e1a49bca8313ddf903d7b1

Summary: FCOS is registered only because we make an import from `get_default_cfg`, if user don't call it (eg. using their own runner), they might find that the meta-arch is not registered.

Reviewed By: ppwwyyxx

Differential Revision: D31920026

fbshipit-source-id: 59eeeb3d1bf30d6b08463c2814930b1cadd7d549

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/130

We want to make sure that after importing `d2go.modeling` all the meta-arch is registered.

Reviewed By: Maninae

Differential Revision: D31904303

fbshipit-source-id: 3f32b65b764b2458e2fb9c4e0bbd99824b37ecfc

Summary:
### New commit log messages
  1f7bd6650 Mark accelerator connector as protected (#10032)

Reviewed By: yifuwang

Differential Revision: D31905981

fbshipit-source-id: a7f0f03033b02b603d28203ae2c8e8df4933fb23

Reviewed By: sstsai-adl

Differential Revision: D31806054

fbshipit-source-id: 4ea98405e1f94176cb77ca69077adf9f4d22e77e

Summary:
### New commit log messages
  0aa220b46 Remove deprecated `distributed_backend` from `Trainer` (#10017)

Reviewed By: kandluis

Differential Revision: D31788128

fbshipit-source-id: 4d9394e119b3122014fc9681a5c56aac8df49141

Summary: this utility function was added in D30272112 (https://github.com/facebookresearch/d2go/commit/737d099b0a8b0fb1f548435e73f95e1252442827) and is useful to all D2 (https://github.com/facebookresearch/d2go/commit/7992f91324aee6ae59795063a007c6837e60cdb8) users as well

Differential Revision: D31833523

fbshipit-source-id: 0adfc612adb8b448fa7f3dbec1b1278c309554c5

Summary: see bottom diff

Reviewed By: newstzpz

Differential Revision: D31781835

fbshipit-source-id: 501b51e7bf92cf3505060a62822fa36f1ed3a7d4