Skip to content

GitLab

Switch branch/tag
  1. 24 Sep, 2021 1 commit
  3. 15 Sep, 2021 1 commit
  5. 09 Sep, 2021 1 commit
  7. 31 Aug, 2021 1 commit
    • Yanghan Wang's avatar
      enable (fake) inference for bolt exported model · e62c0e4c
      Yanghan Wang authored 
      Summary:
Enable the inference for boltnn (via running torchscript).
- merge rcnn's boltnn test with other export types.
- misc fixes.

Differential Revision: D30610386

fbshipit-source-id: 7b78136f8ca640b5fc179cb47e3218e709418d71
      e62c0e4c
  9. 18 Aug, 2021 2 commits
    • Siddharth Shah's avatar
      torch batch boundary CE loss · 7ae35eec
      Siddharth Shah authored 
      Summary:
A torch version which is batched allows us to avoid CPU <--> GPU copy which
gets us ~200ms per iteration saving. This new version of generating boundary
weight mask produces identical masks.

Reviewed By: wat3rBro

Differential Revision: D30176412

fbshipit-source-id: 877f4c6337e7870d3bafd8eb9157ac166ddd588a
      7ae35eec
    • Valentin Andrei's avatar
      Add multi-tensor optimizer version for SGD · 918abe42
      Valentin Andrei authored 
      Summary:
Added multi-tensor optimizer implementation for SGD, from `torch.optim._multi_tensor`. It can potentially provide ~5% QPS improvement by using `foreach` API to speed up the optimizer step.

Using it is optional, from the configuration file, by specifying `SGD_MT` in the `SOLVER.OPTIMIZER` setting.

Reviewed By: zhanghang1989

Differential Revision: D30377761

fbshipit-source-id: 06107f1b91e9807c1db5d1b0ca6be09fcbb13e67
      918abe42
  11. 17 Aug, 2021 1 commit
  13. 16 Aug, 2021 1 commit
  15. 27 Jun, 2021 1 commit
  17. 25 Jun, 2021 1 commit
    • Sam Tsai's avatar
      use src dataset name instead of the derived class name · d4aedb83
      Sam Tsai authored 
      Summary: "@ [0-9]classes" is appended to datasets to mark whether it is a derived class of the original one and saved as a config. When reloading the config, the derived class name will be used as the source instead of the original source. Adding a check to remove the derived suffix.

Reviewed By: wat3rBro

Differential Revision: D29315132

fbshipit-source-id: 0cc204d305d2da6c9f1817aaf631270bd874f90d
      d4aedb83
  19. 21 Jun, 2021 1 commit
    • Yuxin Wu's avatar
      additional flop counting using fvcore's flop counter · bc9d5070
      Yuxin Wu authored 
      Summary:
1. save 3 versions of flop count, using both mobile_cv's flop counter and fvcore's flop counter
2. print only a simple short table in terminal, but save others to files

The `print_flops` function seems not used anywhere so this diff just replaced it.

TODO: enable this feature automatically for train/eval workflows in the next diff

Reviewed By: zhanghang1989

Differential Revision: D29182412

fbshipit-source-id: bfa1dfad41b99fcda06b96c4732237b5e753f1bb
      bc9d5070
  21. 16 Jun, 2021 1 commit
    • Sam Tsai's avatar
      add check/filter for invalid bounding boxes · 692a4fb3
      Sam Tsai authored 
      Summary: Checks for invalid bounding boxes and removes from the being included.

Reviewed By: wat3rBro

Differential Revision: D28902711

fbshipit-source-id: 1f017d6ccf5c959059bcb94a09ddd81de868feed
      692a4fb3
  23. 14 Jun, 2021 1 commit
  25. 01 Jun, 2021 1 commit
    • Yanghan Wang's avatar
      misc update to config utils · 81ab967f
      Yanghan Wang authored 
      Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/77

- Reimplement `get_cfg_diff_table` by reusing other utils
- Adding `reorder` option for `flatten_config_dict`
- Remove the legacy BC support for `ARCH_DEF`, including `str_wrap_fbnet_arch_def` and customized `merge_from_other_cfg`.
- Move `temp_defrost` from `utils.py` to `config.py`, this way there's no more namespace forwarding for `utils.py`
- Merge `test_config_utils.py` and `test_configs.py`

Reviewed By: zhanghang1989

Differential Revision: D28734493

fbshipit-source-id: 925f5944cf0e9019e4c54462e851ea16a5c94b8c
      81ab967f
  27. 25 May, 2021 2 commits
  29. 22 May, 2021 1 commit
  31. 21 May, 2021 2 commits
    • Sanjeev Kumar's avatar
      Enable inference config in export step · 90aff5da
      Sanjeev Kumar authored 
      Summary:
- Enable sdk inference config specification in export step. This enables adding the sdk configuration as part of model file in the export step. The sdk config can be specified as infernece_config.yaml and is zipped together with torchscript model. The main goal of sdk configuration is to control the model inference behavior with model.
- SDK inference config design doc: https://docs.google.com/document/d/1j5qx8IrnFg1DJFzTnu4W8WmXFYJ-AgCDfSQHb2ACJsk/edit
- One click fblearner pipeline is in next diff on the stack

Differential Revision: D27881742

fbshipit-source-id: 34a3ab7a88f456b74841cf671ea1b3f678cdb733
      90aff5da
    • Sam Tsai's avatar
      adding bounding box only options · 27bef8e3
      Sam Tsai authored 
      Summary: Option to change only bounding boxes, others remain the same.

Differential Revision: D28339388

fbshipit-source-id: 7a6d4c5153cf10c473992119f4c684e0b9159b44
      27bef8e3
  33. 17 May, 2021 1 commit
    • Kai Zhang's avatar
      add dataset visualization · 536e9d25
      Kai Zhang authored 
      Summary: Add dataset visualization so that we could visualize test results in Tensorboard.

Reviewed By: zhanghang1989

Differential Revision: D28457363

fbshipit-source-id: 4c2fd9dce349c6fb9e1cec51c9138cf0abb45d7b
      536e9d25
  35. 12 May, 2021 1 commit
    • Luis Perez's avatar
      Synchronize PyTorchLightning/pytorch-lightning (revision 7b283e3c@master) to... · 0848c589
      Luis Perez authored 
      Synchronize PyTorchLightning/pytorch-lightning (revision 7b283e3c@master) to github/third-party/PyTorchLightning/pytorch-lightning

Summary:
# Manual
 - remove fixme's in `model_checkpoint.py`, `parameter_monitor.py`, `test_quantization.py`, and `speed_monitor.py` now that `Trainer` is properly annotated.
- update `test_quantization.py` to `trainer.train_loop.global_step` instead of `trainer.global_step` which is a read-only.
- update `loop_callback.py` to read from `train_loop` for `batch_idx` (which is no longer available).

# Automatic
### New commit log messages
  7b283e3c Bugfix/Multiple dataloaders (#7433)
  d7c44cc6 Docs: sync chlog 1.3.1 (#7478)
  fdf50a5e Mark certain Trainer APIs as protected (#7420)
  ad9118f0 remove trainer hidden state | sanity refactor [1 / n] (#7437)
  4a1134db Log epoch metrics before firing the `on_evaluation_end` hook (#7272)
  b65ae794 Automatically check `DataModule.has_{setup,teardown,prepare_data}` [2/2] (#7238)
  8660d8cf [pre-commit.ci] pre-commit autoupdate (#7475)
  f6fe715e Fix Sphinx argument deprecation (#7464)

Reviewed By: shuyingsunshine21

Differential Revision: D28353491

fbshipit-source-id: 98b87d99e2f09b47b07270858fcbdb5d5299730b
      0848c589
  37. 07 May, 2021 1 commit
  39. 05 May, 2021 1 commit
    • Sam Tsai's avatar
      add enlarge bounging box manipulation · e1961ad4
      Sam Tsai authored 
      Summary: Add a bounding manipulation tool to padding bounding box data.

Reviewed By: newstzpz

Differential Revision: D28082071

fbshipit-source-id: f168cae48672c4fa5c4ec98697c57ed7833787ab
      e1961ad4
  41. 04 May, 2021 1 commit
  43. 30 Apr, 2021 1 commit
    • Sam Tsai's avatar
      add keypoints metadata registry · 77ebe09f
      Sam Tsai authored 
      Summary:
1. Add a keypoint metadata registry for registering different keypoint metadata
2. Add option to inject_coco_dataset for adding keypoint metadata

Reviewed By: newstzpz

Differential Revision: D27730541

fbshipit-source-id: c6ba97f60664fce4dcbb0de80222df7490bc6d5d
      77ebe09f
  45. 28 Apr, 2021 1 commit
    • Ananth Subramaniam's avatar
      Synchronize PyTorchLightning/pytorch-lightning (revision 7fe8d184@master) to... · a95c7983
      Ananth Subramaniam authored 
      Synchronize PyTorchLightning/pytorch-lightning (revision 7fe8d184@master) to github/third-party/PyTorchLightning/pytorch-lightning

Summary:
### New commit log messages
  7fe8d184 Do not `shuffle` in `LightningDataModule.from_datasets` for `IterableDataset` (#7053)
  bab72255 [fix] Add barriers before and after setup hook is run (#7202)
  f920ba29 [bugfix] Metric not logged properly in manual optimization (#7228)
  e147127c [feat] Add better support for predict + ddp 2/3 (#7215)
  ca6c87ff Add back `clip_gradients(model)` (#7231)
  3b36d81c Fixed `num_sanity_val_steps` affecting reproducibility of training data shuffling (#7014)
  5cf9afa1 Add fairscale install msg for Sharded Plugins (#7213)
  52a5cee0 Set smarter default for DDP sharded for performance optimization (#6937)
  dd5ec75e Deprecate save_function from model checkpoint callback (#7201)
  ac7d6a35 Fix `NeptuneLogger.log_text(step=None)` (#7194)
  6be0a859 Update teardown for TPU acc (#7211)
  bc3f08b0 [fix] Add barrier to accelerator's teardown (#6814)
  68eac4d9 Enforce Lightning module as source of truth for automatic optimization (#7130)
  44d775fc Update Error message for ProfileConnector (#7204)
  31fcd7d0 Deprecate write_predictions on the LightningModule (#7066)
  591b9cee make bug_report_model minimal (#7191)
  b3fe8366 Move metrics_to_scalars to a dedicated utilities file (#7180)
  f58865aa Properly set `LightningModule.device` after model replacement (#7188)
  8439aead Update FairScale on CI (#7017)
  92af3632 Fix `lr_finder` suggesting too high learning rates (#7076)
  d534e53e add missing predict docs (#7150)

Reviewed By: kazhang

Differential Revision: D28032962

fbshipit-source-id: 18cd01e8ecc13fe25f0890ac0f4b20c3c3e1fed3
      a95c7983
  47. 21 Apr, 2021 1 commit
  49. 20 Apr, 2021 1 commit
  51. 19 Apr, 2021 1 commit
    • Peizhao Zhang's avatar
      Added hooks to report training progress to fblearner and keep alive. · bd6043ee
      Peizhao Zhang authored 
      Summary:
* Added a registry to register functions that could be used to register hooks for training.
  * TRAINER_HOOKS_REGISTRY: List of functions to add hooks for trainer, all functions in the registry will be called to add hooks
  * `func(hooks: List[HookBase]) -> None`

Reviewed By: zhanghang1989

Differential Revision: D27560806

fbshipit-source-id: fcfa02623bfd08508b6083db2d318d08f7e3c0b8
      bd6043ee
  53. 17 Apr, 2021 1 commit
    • Kai Zhang's avatar
      Delegate to model's customization · aeb24a92
      Kai Zhang authored 
      Summary: Delegate FX quantization callback's customization to model.

Reviewed By: wat3rBro

Differential Revision: D27669212

fbshipit-source-id: 2715546cf03134896da6f95ecddaf8503ff95d0b
      aeb24a92
  55. 15 Apr, 2021 1 commit
  57. 14 Apr, 2021 1 commit
  59. 09 Apr, 2021 2 commits
  61. 05 Apr, 2021 1 commit
    • Owen Wang's avatar
      make prediction count evaluation stable with DDP · cbdd46ab
      Owen Wang authored 
      Summary: Prediction count evaluator needs to gather it's state before computing metrics, otherwise when parallelized across N GPUs, we only get metrics computed from 1/N of the dataset, increasing our eval signal's variance.

Reviewed By: wat3rBro

Differential Revision: D27416864

fbshipit-source-id: b2c5334cd5a38bebcd06c6ace1627a6b71645fdd
      cbdd46ab
  63. 31 Mar, 2021 2 commits
    • Kai Zhang's avatar
      Sync quantization callback changes · 1850a632
      Kai Zhang authored 
      Reviewed By: newstzpz

Differential Revision: D27255960

fbshipit-source-id: 1699ff23d2bc610dffc0215a90a7c1c17e3783c3
      1850a632
    • Sam Tsai's avatar
      fix bypassed rebase error · 1027896a
      Sam Tsai authored 
      Summary: Fixing unit test that was not listed due to rebase error.

Reviewed By: newstzpz, wat3rBro

Differential Revision: D27456322

fbshipit-source-id: 519c5c086adfb19104ed99234f4f476eb34a79bc
      1027896a
  65. 30 Mar, 2021 1 commit
    • Sam Tsai's avatar
      reorganize unit tests · a0658c4a
      Sam Tsai authored 
      Summary: Separate unit tests into individual folder based on functionality.

Reviewed By: wat3rBro

Differential Revision: D27132567

fbshipit-source-id: 9a8200be530ca14c7ef42191d59795b05b9800cc
      a0658c4a
  67. 24 Mar, 2021 2 commits
    • Kai Zhang's avatar
      Support evaluate predictor · 6aec097e
      Kai Zhang authored 
      Summary:
Evaluate the predictor generated by previous step.
This diff modify the lightning_train_net to reuse the evaluation logic by adding a `predictor_path` param.
This diff also makes Lightning training backend depends on `cfg.MODEL.DEVICE` so that in evaluate_predictor step, user could set backend by changing model device. This is useful for evaluating int8 quantized model.

Reviewed By: newstzpz

Differential Revision: D27150609

fbshipit-source-id: fb72da3e81db932c0fa479350150720143e09a3e
      6aec097e
    • Kai Zhang's avatar
      Simplify Lightning task and model creation · 9051f71a
      Kai Zhang authored 
      Summary:
Given that the way to create D2 (https://github.com/facebookresearch/d2go/commit/465cdb842513eb910aa20fcedea1d2edd15dc7b7)go runner and Lightning task are different, get_class was introduced so that in application we could do:
```
if is Lightning:
    task_cls = get_class(classname)
    task = task_cls(cfg)
else:
    runner = create_runner(classname)
```
It turns out that we could need to do that in many places: workflow, binaries.
This diff revert `get_class` and return class in `create_runner` if the class is a Lightning module.

Reviewed By: newstzpz

Differential Revision: D26676595

fbshipit-source-id: c3ce2016d09fe073af4c2dd9f98eea4e59ca621b
      9051f71a