GitLab

Summary: Allow string name of export type to indicate which mobile opt backend user wants to trigger.

Reviewed By: wat3rBro

Differential Revision: D35375928

fbshipit-source-id: dc3f91564681344e1d43862423ab5dc63b6644d3

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/200

Currently when exporting the RCNN model, we call it with `self.model.inference(inputs, do_postprocess=False)[0]`, therefore the output of exported model is not post-processed, eg. the mask is in the squared shape. This diff adds the option to include postprocess in the exported model.

Worth noting that since the input is a single tensor, the post-process doesn't resize the output to original resolution, and we can't apply the post-process twice to further resize it in the Predictor's PostProcessFunc, add an assertion to raise error in this case. But this is fine for most production use cases where the input is not resized.

Set `RCNN_EXPORT.INCLUDE_POSTPROCESS` to `True` to enable this.

Reviewed By: tglik

Differential Revision: D34904058

fbshipit-source-id: 65f120eadc9747e9918d26ce0bd7dd265931cfb5

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/199

- `create_fake_detection_data_loader` currently doesn't take `cfg` as input, sometimes we need to test the augmentation that needs more complicated different cfg.
- name is a bit bad, rename it to `create_detection_data_loader_on_toy_dataset`.
- width/height were the resized size previously, we want to change it to the size of data source (image files) and use `cfg` to control resized size.

Update V3:
In V2 there're some test failures, the reason is that V2 is building data loader (via GeneralizedRCNN runner) using actual test config instead of default config before this diff + dataset name change. In V3 we uses the test's runner instead of default runner for the consistency. This reveals some real bugs that we didn't test before.

Reviewed By: omkar-fb

Differential Revision: D35238890

fbshipit-source-id: 28a6037374e74f452f91b494bd455b38d3a48433

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/201

Adds profiler registry.

Reviewed By: Maninae, wat3rBro

Differential Revision: D34725664

fbshipit-source-id: 52cb99b618e5ba5f9bd8d272d4dcaa770d66983a

Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/198

Differential Revision: D35223296

fbshipit-source-id: 098fc148bc1cea59490cd70ef6f122062b2de7a6

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/195

Update:
The problem is solved by: https://github.com/pytorch/pytorch/issues/74779
This diff only adds the step to show the env, so it's easier to report bug in the future.

Reviewed By: tglik

Differential Revision: D35150748

fbshipit-source-id: 1f648ec5abb9f3563445cca26cc53b5f7e03bf2c

Reviewed By: wat3rBro

Differential Revision: D34813614

fbshipit-source-id: e062cf5e1cbee09bb1970a21d21bc4132aec0890

Summary: Tweak exporter and evaluator cli entry point func to support calling it as a module with args from custom launching code.

Reviewed By: sstsai-adl

Differential Revision: D35035813

fbshipit-source-id: c8b24099e94ccc58c184f8aac95b2a24a137e86a

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/191

When exporting model to torchscript (using `MODEL.DEVICE = "cpu"`), mean/std are constant instead of model parameters. Therefore after casting the torchscript to CUDA, the mean/std remains on cpu. This will cause problem when running inference on GPU.

The fix is exporting the model with `MODEL.DEVICE = "cuda"`. However D2 (https://github.com/facebookresearch/d2go/commit/87374efb134e539090e0b5c476809dc35bf6aedb)Go internally uses "cpu" during export (via cli: https://fburl.com/code/4mpk153i, via workflow: https://fburl.com/code/zcj5ud4u) by default. For CLI, user can manually set `--device`, but for workflow it's hard to do so. Further more it's hard to support mixed model using single `--device` option. So this diff adds a special handling in the RCNN's `default_prepare_for_export` to bypass the `--device` option.

Reviewed By: zhanghang1989

Differential Revision: D35097613

fbshipit-source-id: df9f44f49af1f0fd4baf3d7ccae6c31e341f3ef6

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/193

- Add daily CI.
- rename `Unit Test` to `CI`.
- rename the `build` to `python-unittest`.

Reviewed By: tglik

Differential Revision: D35102575

fbshipit-source-id: cbcc3992e019d3e00d8e862b81de4bb1c16f8fe1

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/192

Nowadays lightning will initialize process group when using ddp strategy, since `TestLightningTrainNet` does a training with ddp strategy (https://fburl.com/code/a9yp0kzy), the process group ended up initialized after running the test. However there're other tests that will also set up ddp and thus expect non-initialized process group, this is not a problem on sandcastle since the tests run separately, however in OSS env, the tests are running together, so the error happens (eg. https://github.com/facebookresearch/d2go/runs/5668912203?check_suite_focus=true).

This diff adds a clean up step in `TestLightningTrainNet`.

Reviewed By: tglik

Differential Revision: D35099944

fbshipit-source-id: f5b42b2a87d4efd9aa0ed97e6bd2140d80ab9522

Summary: Detectron2[Go]'s Visualizer and sem_seg_evaluation now updated with customization entrypoints for how to handle reading semantic seg masks. By default, PIL and PNG images are expected. However, some specific projects' datasets use .npy files and this customization allows providing an alternate Visualizer and evaluation function for reading them.

Reviewed By: newstzpz

Differential Revision: D33434948

fbshipit-source-id: 42af16d6708ffc5b2c03ec8507757313e23c8204

Summary: rm TARGET in gitignore

Reviewed By: newstzpz

Differential Revision: D35014854

fbshipit-source-id: 4a28f797bd5277eb58df6921f3ae9b7debb65f71

Summary: Add documentation on the pre and post processing functions for segmentation.

Reviewed By: XiaoliangDai

Differential Revision: D34882165

fbshipit-source-id: 375c62d0ad632a40b6557065b3362e333df8c55f

Summary:
- remove the `None` support for `merge_from_list`
- fix logging when initializing diskcacje
- Don't inherit `_FakeListObj` fron `list`, so looping over it can raise error.

Reviewed By: sstsai-adl

Differential Revision: D34952714

fbshipit-source-id: f636e408b79ed77904f257f189fcef216cb2efbc

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/190

Currently there is some fragmentation in export for how to apply convert logic in various mode. `prepare_for_quant_convert` is only called in non eager modes and the logic in eager mode is not customizable.
This diff unify the `prepare_for_quant_convert` code path for all quantization modes.
Also in this diff we rename `_non_qat_to_qat_state_dict_map`, that is use in qat checkpointer to be publish var `non_qat_to_qat_state_dict_map` and allow models to populate it with custom mapping. This is useful in cases where the param mapping between the non qat model and the qat model cannot be inferred definitely (see note in https://fburl.com/code/9rx172ht) and have some ambiguity that can only be resolved by the model logic.

Reviewed By: wat3rBro

Differential Revision: D34741217

fbshipit-source-id: 38edfec64200ec986ffe4f3d47f527cb6a3fb5e9

Summary:
D33301363 changes the signature of `update_cfg` from `update_cfg(cfg, *updaters)` to `update_cfg(cfg, updaters, new_allowed)`, while the call sites are not updated. Eg. https://www.internalfb.com/code/fbsource/[9e071979a62ba7fd3d7a71dee1f0809815cbaa43]/fbcode/fblearner/flow/projects/mobile_vision/detectron2go/core/workflow.py?lines=221-225, the `merge_from_list_updater(e2e_train.overwrite_opts),` is then not used.

For the fix:
- Since there're a lot of call sites for `update_cfg` it's better to keep the original signature.
- ~~~The `new_allowed` can actually be passed to each individual updater instead of the `update_cfg`, this also gives finer control.~~~
- Make override the `merge_from_list` to make it respect `new_allowed`.
- Preserve the `new_allowed` for all nodes (not only the root) in the FLOW Future calls.

Reviewed By: zhanghang1989

Differential Revision: D34840001

fbshipit-source-id: 14aff6bec75a8b53d4109e6cd73d2494f68863b4

Summary:
Dataloader:
Rewrote the data loader via build_stream_dataset_reader with the DATASET_DEFINITION of "peopleai_face_eng_inference_results".

User Calibration Model (initial version):

nn.Sequential(
    nn.Conv1d(72, 128, 1),
    nn.BatchNorm1d(128),
    nn.ReLU(),
    nn.Flatten(),
    nn.Linear(128, 72),
)

Differential Revision: D34202009

fbshipit-source-id: 55a2c579e463ed19eac38b5dd12e11c09cbccc11

Reviewed By: kazhang

Differential Revision: D34669519

fbshipit-source-id: 8cfee968104c823a55960f2730d8e888ac1f298e

Summary:
see https://fb.workplace.com/notes/3006074566389155

----
did the integration test not catch this?

Reviewed By: ananthsub, tangbinh

Differential Revision: D34665501

fbshipit-source-id: ff2cbfa9462f131455dce46a0c413c4c69105f48

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/189

X-link: https://github.com/facebookresearch/recipes/pull/14

X-link: https://github.com/facebookresearch/ReAgent/pull/616

### New commit log messages
- [9b011606f Add callout items to the Docs landing page (#12196)](https://github.com/PyTorchLightning/pytorch-lightning/pull/12196)

Reviewed By: edward-io

Differential Revision: D34687261

fbshipit-source-id: 3ef6be5169a855582384f9097a962d2261625882

Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/187

Reviewed By: ananthsub, zhanghang1989

Differential Revision: D34650467

fbshipit-source-id: b9518e5dd673b709320b87e57a43d053eca3aabe

Reviewed By: tangbinh

Differential Revision: D34669294

fbshipit-source-id: c87bc1d4c589518f7c9fc21e6dfe27b03e700b6d

Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/186

Reviewed By: zhanghang1989

Differential Revision: D34648894

fbshipit-source-id: 51e4ea978b84a81f7e4dd91800b75a916da08faa

Summary: fix D34540275 (https://github.com/facebookresearch/d2go/commit/d8bdc633ec66e6ce73076d027f8e777791c2e067)

Reviewed By: tglik

Differential Revision: D34662745

fbshipit-source-id: 6fd67db041fab6f5810763702e4cc3f16a08c5df

Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/188

Reviewed By: tangbinh, wat3rBro

Differential Revision: D34658350

fbshipit-source-id: 36e8c1e8c5dab97990b1d9a5b1a58667e0e3c455

Summary:
### New commit log messages
- [7e2f9fbad Refactor codebase to use `trainer.loggers` over `trainer.logger` when needed (#11920)](https://github.com/PyTorchLightning/pytorch-lightning/pull/11920)

Reviewed By: edward-io

Differential Revision: D34583686

fbshipit-source-id: 98e557b761555c24ff296fff3ec6881d141fa777

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/185

The `DiskCachedDatasetFromList` was originally in the `d2go/data/utils.py`, so the class is declared by default. Therefore the clean up call (https://fburl.com/code/cu7hswhx) is always called even when the feature is not enabled. This diff move it to a new place and delay the import, so the clean up won't run.

Reviewed By: tglik

Differential Revision: D34601363

fbshipit-source-id: 734bb9b2c7957d7437ad40c4bfe60a441ec2f23a

Summary: Add option for controlling empty annotation filtering.

Reviewed By: zhanghang1989

Differential Revision: D34365265

fbshipit-source-id: 261c6879636f19138de781098f47dee4909de9e7

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/179

Refactored extended coco to fix lint errors and also simpler error reporting.

Differential Revision: D34365252

fbshipit-source-id: 8bf221eba5b8c5e63ddcf5ca19d7486726aff797

Summary:
Add support in d2go.distributed for `env://` init method. Use env variables as specified in https://pytorch.org/docs/stable/distributed.html#environment-variable-initialization for initialized distributed params.

Also change train_net cli function signature to accept args list instead of only using `sys.argv`. To allow calling this function from AIEnv launcher.

Differential Revision: D34540275

fbshipit-source-id: 7f718aed4c010b0ac8347d43b5ca5b401210756c

Summary:
`Detectron2GoRunner` defaults to trigger an evaluation right after the last iteration in the `runner.do_train` method. This sometimes might be unnecessary, because there is a `runner.do_test` at the end of training anyways.

It could also lead to some side effects. For example, it would cause the training and test data loader present at the same time, which led to an OOM issue in our use case.

In this diff, we add an option `eval_after_train` in the `EvalHook` to allow users to disable the evaluation after the last training iteration.

Reviewed By: wat3rBro

Differential Revision: D34295685

fbshipit-source-id: 3612eb649bb50145346c56c072ae9ca91cb199f5

Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/184

Reviewed By: zhanghang1989

Differential Revision: D34529248

fbshipit-source-id: f77882dae7de336da77ac9bb7c35cfc1e8d541af

Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/183

Reviewed By: zhanghang1989

Differential Revision: D34492204

fbshipit-source-id: 7fd459172e83a5015ca9eee0e2018ce8b22c3096

Summary:
# TLDR: To use this feature, setting `D2 (https://github.com/facebookresearch/d2go/commit/7992f91324aee6ae59795063a007c6837e60cdb8)GO_DATA.DATASETS.DISK_CACHE.ENABLED` to `True`.

To support larger datasets, one idea is to offload the DatasetFromList from RAM to disk to avoid OOM. `DiskCachedDatasetFromList` is a drop-in replacement for `DatasetFromList`, during `__init__`, it puts serialized list onto the disk and only stores the mapping in the RAM (the mapping could be represented by a list of addresses or even just a single number, eg. every N item is grouped together and N is the fixed number), then the `__getitem__` reads data from disk and deserializes the element. Some more details:
- Originally the RAM cost is `O(s*G*N)` where `s` is average data size, `G` is #GPUs, `N` is dataset size. When diskcache is enabled, depending on the type of mapping, the final RAM cost is constant or O(N) with a very small coefficient; the final disk cost is `O(s*N)`.
- The RAM usage is peaked at preparing stage, the cost is `O(s*N)`, if this becomes bottleneck, we probably need to think about modifying the data loading function (registered in DatasetCatalog). We also change the data loading function to only run on local master process, otherwise RAM will be peaked at `O(s*G*N)` if all processes are loading data at the same time.
- The time overhead of initialization is linear to dataset size, this is capped by disk I/O speed and performance of diskcache library. Benchmark shows it can at least handle 1GB per minute if writing in chucks (much worse if not), which should be fine in most use cases.
- There're also a bit time overhead when reading the data, but this is usually negligible compared with reading files from external storage like manifold.

It's not very easy to integrate this into D2 (https://github.com/facebookresearch/d2go/commit/7992f91324aee6ae59795063a007c6837e60cdb8)/D2 (https://github.com/facebookresearch/d2go/commit/7992f91324aee6ae59795063a007c6837e60cdb8)Go cleanly without patching the code, several approaches:
- Integrate into D2 (https://github.com/facebookresearch/d2go/commit/7992f91324aee6ae59795063a007c6837e60cdb8) directly (modifying D2 (https://github.com/facebookresearch/d2go/commit/7992f91324aee6ae59795063a007c6837e60cdb8)'s `DatasetFromList` and `get_detection_dataset_dicts`): might be the cleanest way, but D2 (https://github.com/facebookresearch/d2go/commit/7992f91324aee6ae59795063a007c6837e60cdb8) doesn't depend on `diskcache` and this is a bit experimental right now.
- D2 (https://github.com/facebookresearch/d2go/commit/7992f91324aee6ae59795063a007c6837e60cdb8)Go uses its own version of [_train_loader_from_config](https://fburl.com/code/0gig5tj2) that wraps the returned `dataset`. It has two issues: 1): it's hard to make the underlying `get_detection_dataset_dicts` only run on local master, partly because building sampler uses `comm.shared_random_seed()`, things can easily go out-of -sync 2): needs some duplicated code for test loader.
- pass new arguments along the way, it requires touching D2 (https://github.com/facebookresearch/d2go/commit/7992f91324aee6ae59795063a007c6837e60cdb8)'s code as well, and we need to carry new arguments in lot of places.

Lots of TODOs:
- Automatically enable this when dataset is larger than certain threshold (need to figure out how to do this in multiple GPUs, some communication is needed if only local master is reading the dataset).
- better cleanups
- figure out the best way of integrating this (patching is a bit hacky) into D2 (https://github.com/facebookresearch/d2go/commit/7992f91324aee6ae59795063a007c6837e60cdb8)/D2 (https://github.com/facebookresearch/d2go/commit/7992f91324aee6ae59795063a007c6837e60cdb8)Go.
- run more benchmarks
- add unit test (maybe also enable integration tests using 2 nodes 2 GPUs for distributed settings)

Reviewed By: sstsai-adl

Differential Revision: D27451187

fbshipit-source-id: 7d329e1a3c3f9ec1fb9ada0298a52a33f2730e15

Summary: It's possible to have `lib` under core `mobile-vision/d2go/{d2go,projects}`, exclude them from `.gitignore`.

Reviewed By: zhanghang1989

Differential Revision: D34288538

fbshipit-source-id: 7094cdf4f52263fbf6ff6707d487bc3328fbbd8b

Summary: We proactively remove references to the deprecated DDP accelerator to prepare for the breaking changes following the release of PyTorch Lighting 1.6 (see T112240890).

Differential Revision: D34295318

fbshipit-source-id: 7b2245ca9c7c2900f510722b33af8d8eeda49919

Summary:
Pull Request resolved: https://github.com/facebookresearch/mobile-vision/pull/61

Pull Request resolved: https://github.com/facebookresearch/d2go/pull/177

Adhoc datasets currently use default register functions. Changed to checking if it was registered in a look up table for injected coco and just using that instead.

Differential Revision: D33489049

fbshipit-source-id: bcb12bba49749a875ea80ae61f4eecc4a5d1e31a

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/180

Distributed backend is deprecated. Switching to use "use_ddp" instead.

Reviewed By: kazhang

Differential Revision: D34394993

fbshipit-source-id: a5bfb22f8952d20c9a8d86322cd740534c25c689

Summary:
Currently, the exporter method takes in a compare_accuracy parameter, which after all the compute (exporting etc.) raises an exception if it is set to True.

This looks like an antipattern, and causes a waste of compute.

Therefore, I am proposing to raise the exception at the very beginning of method call to let the client know in advance that this argument's functionality isn't implemented yet.

NOTE: We might also choose to get rid of the entire parameter. I am open for suggestions.

Differential Revision: D34186578

fbshipit-source-id: d7fbe7589dfe2d2f688b870885ca61e6829c9329