GitLab

Summary:
The dict `record` might not have keys `"width"` or `"height"`.
This diff check if `"width"` and `"height"` are in the dict `record` before getting the values.

Reviewed By: sstsai-adl

Differential Revision: D29243341

fbshipit-source-id: a1e0e343dd1afcced834c3732e64bb6f372fbd1a

Synchronize PyTorchLightning/pytorch-lightning (revision f7459f53@master) to github/third-party/PyTorchLightning/pytorch-lightning

Summary:
## OSS
Note these issues are being solved in OSS here: https://github.com/PyTorchLightning/pytorch-lightning/pull/7994/files#

## Manual
- `speed_monitor.py` - `Result.unpack_batch_size` has been removed, moved to new implementation.
- `fully_sharded.py` - There was a refactor for plugins, so updated corresponding function to keep reduced memory usage.
- `hive_writing_classy.py`, `hive_writing_faim.py`, `hive_writing_xrayvideo.py` - Same as `speed_monitor.py`.
- [Temporary] Uncommented misconfiguration exception. See https://github.com/PyTorchLightning/pytorch-lightning/pull/7882#pullrequestreview-683282719.
- Update `TestModel` to detach appropriately.
- Manually `detach` metrics stored in ResultStore.

## Automatic
### New commit log messages
  f7459f53 DeepSpeed Infinity Update (#7234)
  03e7bdf8 Improve `LightningModule` hook tests (#7944)
  3a0ed02b Properly handle parent modules w/ parameters in `BaseFinetuning` callback (#7931)
  ce93d8bc Handle errors due to uninitailized parameters (#7642)
  cca0e753 remove parsing comments (#7958)
  898fb56b added on_test_start() documentation (#7962)
  22d82661 Seed all workers when using DDP (#7942)
  436fc53c Improve `LightningDataModule` hook test and fix `dataloader_idx` argument (#7941)
  6b7b4047 deprecate hpc_load() and integrate it with restore() (#7955)
  20a5e09e fix myst-parser warning blocking docs ci (#7967)
  f15ea601 update chlog + legacy chpt (#7954)
  59d0c656 Add dataclass support to `apply_to_collection` (#7935)
  cdd01f32 LightningCLI support for argument links applied on instantiation (#7895)
  6856cced Remove rank_zero_only on DataModule prepare_data (#7945)
  96433d03 IPU Integration 5/5 (#7867)
  42c7f272 refactor checkpoint loading for training type plugins (#7928)
  ac4eb0a0 `is_overridden` improvements (#7918)
  9e932f4d Delete `on_after_backward` unused argument (#7925)
  8b738693 Deprecate the default `EarlyStopping` callback monitor value (#7907)
  c1eac483 split `restore_training_state` into logical parts [2 / 2] (#7900)
  d209b689 split `restore_training_state` into logical parts [1 / 2] (#7901)
  111287b4 add pre-commit hooks (#7906)
  839019a3 Remove legacy teardown check in train loop (#7917)
  b45a89a2 Clean-up after logger connector redesign 2/2 (#7631)
  07b69231 Remove fn check for ipu output (#7915)
  580a3b5e Remove dead code (#7910)
  df812398 Clean-up after logger connector redesign 1/2 (#7909)
  ec4f8856 Enable logger connector re-design (#7891)
  15be9865 add logger to __all__ (#6854)
  6fee9262 Deprecate `LightningDataModule` lifecycle properties (#7657)
  764d2c77 refactor CheckpointConnector.restore_weights  (#7862)
  7f4ef6d1 Fix logs overwriting issue for remote fs (#7889)
  c310ce66 Logger connector re-design `_Metadata.reduce_fx` fixes. (#7890)
  b214442e New logger connector code (#7882)

Reviewed By: yifuwang

Differential Revision: D29105294

fbshipit-source-id: 990b2a4a7333908d676de193f5ec930cb50b8a19

Summary: This diff logs D2 (https://github.com/facebookresearch/d2go/commit/692a4fb3c506aeebbb49070a20d139d617381b19)Go model instantiation events to table scuba_caffe2_pytorch_usage_stats, so that we could track model usage in fblearner, bento, local scripts, etc.

Reviewed By: zhanghang1989

Differential Revision: D28986723

fbshipit-source-id: 3e865354e5884c9e82bd1b08819cc10d349f93bd

Summary:
1. Circular pattern segmentation points
2. Use circular pattern for kp patterns

Reviewed By: wat3rBro

Differential Revision: D29069224

fbshipit-source-id: c4c01d6d93de5abbdfceae07f1cd48fb56e05f57

Summary: Checks for invalid bounding boxes and removes from the being included.

Reviewed By: wat3rBro

Differential Revision: D28902711

fbshipit-source-id: 1f017d6ccf5c959059bcb94a09ddd81de868feed

Summary: As titled.

Reviewed By: zhanghang1989

Differential Revision: D29075952

fbshipit-source-id: 6ef3dc35cd436c1fffb031ea59f20ca23afc5368

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/83

- Implement `prepare_for_export` for `SemanticSegmentor`
- Add unit test comparing numerical matching

Reviewed By: zhanghang1989

Differential Revision: D29088421

fbshipit-source-id: ccb86ac4b4b90a63eeebdbf76b2bf31c1da65a8b

Summary:
Major changes
- Add a new runner `EgoDETRRunner` which inherit from existing `DETRRunner` in D2 (https://github.com/facebookresearch/d2go/commit/62c21f252ad314961cf0157ee8f37cc4f7835e1d)GO repo.
- Add a new data mapper `EgoDETRDatasetMapper` which has custom crop transform generator and supports generic data augmentation.

Reviewed By: zhanghang1989

Differential Revision: D28895225

fbshipit-source-id: 4181ff8fce81df22a01d355fdff7e81e83d69e64

Summary: EZ

Reviewed By: zhanghang1989

Differential Revision: D29000628

fbshipit-source-id: f954214dfe3a989fc145663f8bb1870812e78ce7

Summary: Use all training dataset for export instead of just first. This is to support use cases where there is only a small amount of images per jsons but a number of jsons. Since calibration uses the first dataset, it is limited by the number of images in a single dataset.

Reviewed By: ppwwyyxx

Differential Revision: D28902673

fbshipit-source-id: f80146b02d2d1bc04703fbb21ef410f5e26ba64c

Summary: Detectron2 and D2 (https://github.com/facebookresearch/d2go/commit/81ab967feb650145d3a5904f20fdddd28be83445)Go use custom sampler, we don't need Lightning to add distributed sampler.

Reviewed By: ananthsub

Differential Revision: D28921092

fbshipit-source-id: ec8f310d0590ed92227935b979d59a06d7fb7a69

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/77

- Reimplement `get_cfg_diff_table` by reusing other utils
- Adding `reorder` option for `flatten_config_dict`
- Remove the legacy BC support for `ARCH_DEF`, including `str_wrap_fbnet_arch_def` and customized `merge_from_other_cfg`.
- Move `temp_defrost` from `utils.py` to `config.py`, this way there's no more namespace forwarding for `utils.py`
- Merge `test_config_utils.py` and `test_configs.py`

Reviewed By: zhanghang1989

Differential Revision: D28734493

fbshipit-source-id: 925f5944cf0e9019e4c54462e851ea16a5c94b8c

Reviewed By: sanjeevk42

Differential Revision: D28346869

fbshipit-source-id: b226acf5ee5d90be4ea183dc7de92133db4d5717

Summary: Add an option to set the number of test images. Thus, during finetune, we can set a small number of test images (for only visualization purpose) to save the time for evaluation.

Reviewed By: leehomyc

Differential Revision: D28720086

fbshipit-source-id: 8085be6a0f4f8742784e3dafe255716f3ae02acb

Summary: Currently we are checking if MODEL.DEVICE is "gpu", but actually we DEVICE could also be "cuda". This diff checks if device is "cpu" instead.

Reviewed By: wat3rBro

Differential Revision: D28689547

fbshipit-source-id: 7512d32b7c08b0dcdc6487c6c2f1703655e64b19

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/75

Refactor the base test case
- make test_dir valid throughout the test (rather than under local context), so individual test can load back the export model
- refactor the `custom_setup_test` for easier override.
- move parameterized into base class to avoid copying naming function

Reviewed By: zhanghang1989

Differential Revision: D28651067

fbshipit-source-id: c59a311564f6114039e20ed3a23e5dd9c84f4ae4

Summary: Currently when launching a training flow, we read number of processes from resources.num_gpus. To be backward compatible with existing D2 (https://github.com/facebookresearch/d2go/commit/f82d44d3c33e6c781a3c6f2b27b376fdfbaeda53)Go training config, this diff changes to dist_config.num_processes_per_machine instead.

Reviewed By: wat3rBro

Differential Revision: D28630334

fbshipit-source-id: 3c684cd56e5d2e247c7b82e1d1eeff0f39e59ee4

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/76

Detectron2GoRunner doesn't contain configs about exporting RCNN models, use GeneralizedRCNNRunner instead

Reviewed By: zhanghang1989

Differential Revision: D28652627

fbshipit-source-id: 6f324f608d8b2abdf98179a36e4b79837f135340

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/70

DDP supports an fp16_compress_hook which compresses the gradient to FP16 before communication. This can result in a significant speed up.

Add one argument `_C.MODEL.DDP_FP16_GRAD_COMPRESS` to trigger it.

Reviewed By: zhanghang1989

Differential Revision: D28467701

fbshipit-source-id: 3c80865222f48eb8fe6947ea972448c445ee3ef3

Differential Revision:
D27881742 (https://github.com/facebookresearch/d2go/commit/90aff5daf608473dd312b300db8615326fa40a37)

Original commit changeset: 34a3ab7a88f4

fbshipit-source-id: 42c03b4f2b69c656b26774a4665b84b832262650

Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/73

Reviewed By: kazhang

Differential Revision: D28610607

Pulled By: zhanghang1989

fbshipit-source-id: c079bc6fff64cd452db6750b3e984546c47fdfe7

Summary:
- Enable sdk inference config specification in export step. This enables adding the sdk configuration as part of model file in the export step. The sdk config can be specified as infernece_config.yaml and is zipped together with torchscript model. The main goal of sdk configuration is to control the model inference behavior with model.
- SDK inference config design doc: https://docs.google.com/document/d/1j5qx8IrnFg1DJFzTnu4W8WmXFYJ-AgCDfSQHb2ACJsk/edit
- One click fblearner pipeline is in next diff on the stack

Differential Revision: D27881742

fbshipit-source-id: 34a3ab7a88f456b74841cf671ea1b3f678cdb733

Summary: Option to change only bounding boxes, others remain the same.

Differential Revision: D28339388

fbshipit-source-id: 7a6d4c5153cf10c473992119f4c684e0b9159b44

Summary: Add dataset visualization so that we could visualize test results in Tensorboard.

Reviewed By: zhanghang1989

Differential Revision: D28457363

fbshipit-source-id: 4c2fd9dce349c6fb9e1cec51c9138cf0abb45d7b

Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/58344

remove a helper function thats more trouble then its worth.

ghstack-source-id: 129131889

Reviewed By: dhruvbird

Differential Revision: D28460607

fbshipit-source-id: 31bd6c1cc169785bb360e3113d258b612cad47fc

Summary: Create new CfgNode that is consistent with the parent node.

Reviewed By: zhanghang1989

Differential Revision: D28318466

fbshipit-source-id: 38cb84de6bdfec2b283c4d9a1090cad47c118c9c

Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/68

Reviewed By: kazhang

Differential Revision: D28426745

Pulled By: zhanghang1989

fbshipit-source-id: 5b82b03847e5e1d06d1bfcdaf0192d65e9202e77

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/62

Lightning trainer set max step to cfg.SOLVER.MAX_ITER. However, this is the max iteration for all nodes, in multi-node training, we need to scale it down, as well as eval period and other configs.
This diff calls `auto_scale_world_size` before passing the config to trainer.

Reviewed By: wat3rBro

Differential Revision: D28140877

fbshipit-source-id: 2639ae58773a4ec2a0cc59dfefb2f5d9b1afe1a8

Reviewed By: zhanghang1989

Differential Revision: D28346653

fbshipit-source-id: d80a1f824b097c05029edb171739a4928e47e4d8

Synchronize PyTorchLightning/pytorch-lightning (revision 7b283e3c@master) to github/third-party/PyTorchLightning/pytorch-lightning

Summary:
# Manual
 - remove fixme's in `model_checkpoint.py`, `parameter_monitor.py`, `test_quantization.py`, and `speed_monitor.py` now that `Trainer` is properly annotated.
- update `test_quantization.py` to `trainer.train_loop.global_step` instead of `trainer.global_step` which is a read-only.
- update `loop_callback.py` to read from `train_loop` for `batch_idx` (which is no longer available).

# Automatic
### New commit log messages
  7b283e3c Bugfix/Multiple dataloaders (#7433)
  d7c44cc6 Docs: sync chlog 1.3.1 (#7478)
  fdf50a5e Mark certain Trainer APIs as protected (#7420)
  ad9118f0 remove trainer hidden state | sanity refactor [1 / n] (#7437)
  4a1134db Log epoch metrics before firing the `on_evaluation_end` hook (#7272)
  b65ae794 Automatically check `DataModule.has_{setup,teardown,prepare_data}` [2/2] (#7238)
  8660d8cf [pre-commit.ci] pre-commit autoupdate (#7475)
  f6fe715e Fix Sphinx argument deprecation (#7464)

Reviewed By: shuyingsunshine21

Differential Revision: D28353491

fbshipit-source-id: 98b87d99e2f09b47b07270858fcbdb5d5299730b

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/67

When extending RCNN, sometimes we need to also modify `prepare_for_export/quant`. It might be inconvenient to create new meta-arch just for the purpose of modifying `prepare_for_export/quant`, therefore we create the registry so user can change them via config.

Reviewed By: zhanghang1989

Differential Revision: D28308056

fbshipit-source-id: 4f169eb38292a75d15d3b9f44694480eaa9244e0

Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/66

Reviewed By: zhanghang1989

Differential Revision: D28308297

fbshipit-source-id: 7f6effad7d289546b50f7b16739a04c358a22e21

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/59

* We have an internal dependency:
```
d2go/export/logfiledb.py", line 8, in <module>
    from mobile_cv.torch.utils_caffe2.ws_utils import ScopedWS
    ModuleNotFoundError: No module named 'mobile_cv.torch'
```
This cause the failure of unittest on GitHub
https://github.com/facebookresearch/d2go/pull/58/checks?check_run_id=2471727763

* use python 3.8 because another unittest failure on github ci
```
from typing import final
ImportError: cannot import name 'final' from 'typing' (/usr/share/miniconda/lib/python3.7/typing.py)
```

Reviewed By: wat3rBro

Differential Revision: D28109444

fbshipit-source-id: 95e9774bdaa94f622267aeaac06d7448f37a103f

Summary: Add upperbound-model based video evaluation dataset

Reviewed By: yc-fb

Differential Revision: D28122534

fbshipit-source-id: 37cf5ece1c6a5d537b91ae2f7db74a35b97c210d

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/63

As titled.

Reviewed By: zhanghang1989

Differential Revision: D27945627

fbshipit-source-id: 58180ace6ab327a5eb025f3227ae6c22601a3580

Summary: build_traceable_model is deprecated, remove it from default runner.

Reviewed By: zhanghang1989

Differential Revision: D28243469

fbshipit-source-id: e09729dcc083dc273329e1c2f731065578d8ea42

Reviewed By: zhanghang1989

Differential Revision: D28243043

fbshipit-source-id: fae5491d8978bcdb096fcee69846c707cda0ac00

Summary: `augment_model_with_bundled_inputs` can compress the tensor when values are constant, however it requires contiguous layout and `zero_like` can return non-contiguous ones

Reviewed By: zhanghang1989

Differential Revision: D28224987

fbshipit-source-id: 32b13728ff8fadd53412dbf2d59c4b46e92af04a

Summary: Add a bounding manipulation tool to padding bounding box data.

Reviewed By: newstzpz

Differential Revision: D28082071

fbshipit-source-id: f168cae48672c4fa5c4ec98697c57ed7833787ab

Summary:
[WIP] Will add pretrained weights and update model url & scores

build mask head using fbnet builder and retrain weights

Reviewed By: wat3rBro

Differential Revision: D27992340

fbshipit-source-id: a216a99954eb3784438d595cd09cbb19e70ec3c3