- 14 Jun, 2022 1 commit
-
-
Yanghan Wang authored
Summary: X-link: https://github.com/facebookresearch/mobile-vision/pull/83 Pull Request resolved: https://github.com/facebookresearch/d2go/pull/292 Reviewed By: tglik, newstzpz Differential Revision: D37116726 fbshipit-source-id: 9956fe19916521b4fb512207512861afe4f09148
-
- 13 Jun, 2022 2 commits
-
-
Peizhao Zhang authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/291 Allow AUTO_SCALING_METHODS to be empty. Reviewed By: tglik, wat3rBro Differential Revision: D36887643 fbshipit-source-id: 1391ddccfc46389b3f412295fa6540f81bf76368
-
Yanghan Wang authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/287 In previous D36798327 (https://github.com/facebookresearch/d2go/commit/1f45cf04c69c63f51387b073ae931832b7a9510f), we still have 7 files which have exception during bootstrap. Instead of tweaking the the mock logic to support those scenarios, since they're all related to import-time computation, I think it's reasonable to "delay" those computations, therefore this diff introduces the `lazy_func` decorator, which delays it for bootstrap. After this change, now all files are be bootstrapped without exception, next step we'll change `catch_exception` to False and put a unit test on this to prevent committing code that is not bootstrap-able. Reviewed By: tglik Differential Revision: D36814147 fbshipit-source-id: e344a5ccee76b1d9962117d300926a88c066e051
-
- 10 Jun, 2022 3 commits
-
-
John Reese authored
Summary: pyfmt now specifies a target Python version of 3.8 when formatting with black. With this new config, black adds trailing commas to all multiline function calls. This applies the new formatting as part of rolling out the linttool-integration for pyfmt. paintitblack Reviewed By: zertosh, lisroach Differential Revision: D37084377 fbshipit-source-id: 781a1b883a381a172e54d6e447137657977876b4
-
Yanghan Wang authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/286 Support cache for registry bootstrap. Reviewed By: tglik Differential Revision: D36798475 fbshipit-source-id: 5f32093d6d8d1d1db896d6b2567ddd0187d66248
-
Yanghan Wang authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/284 X-link: https://github.com/facebookresearch/mobile-vision/pull/78 Support setting global timeout and helper function to custom timeout locally. Reviewed By: newstzpz, Cysu Differential Revision: D36797768 fbshipit-source-id: cbf6d52b6b5296f3ee3f6512c467b957c440b025
-
- 09 Jun, 2022 1 commit
-
-
Yanghan Wang authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/274 X-link: https://github.com/facebookresearch/mobile-vision/pull/76 TLDR: this diff consolidate the `distributed_helper` of `mobile_cv`, it (together with `mobile_cv`'s `comm` module) should be the TOGO library for dealing with DDP. D2 (https://github.com/facebookresearch/d2go/commit/87374efb134e539090e0b5c476809dc35bf6aedb)Go's `distributed` is now built on-top of `mobile_cv`'s `distributed_helper`. Reviewed By: newstzpz Differential Revision: D36787336 fbshipit-source-id: 640c9dcff5eec534e7894c75cfdf0a12d21c297e
-
- 08 Jun, 2022 1 commit
-
-
Yanghan Wang authored
Summary: X-link: https://github.com/facebookresearch/mobile-vision/pull/81 Pull Request resolved: https://github.com/facebookresearch/d2go/pull/283 - add `MoreMagicMock`, which handles inheritance and comparison. - also support lazy registering mocked objects (has to be `MoreMagicMock`). - don't need to skip `skip files that doesn't contain ".register()" call` since we can handle most files pretty well now. - also mock the open - delay the import for `from detectron2.utils.testing import assert_instances_allclose`; for some reason python is doing magic things if you import anything starting with `assert`, so the mocked import doesn't work. - makes log function nicer. Reviewed By: tglik Differential Revision: D36798327 fbshipit-source-id: ccda7e7583b95a24f3dde1bbe0468593dacb8663
-
- 07 Jun, 2022 1 commit
-
-
Yanghan Wang authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/281 Reviewed By: tglik Differential Revision: D36414433 fbshipit-source-id: fc9aa5ff23b0a8cdc4ff3acdc8438a0577bf65ec
-
- 06 Jun, 2022 3 commits
-
-
Anthony Dito authored
Summary: Looks like an import changed, this changes it in the ipynb notebook example. > Attached screenshot of this working now in collab.  Pull Request resolved: https://github.com/facebookresearch/d2go/pull/276 Reviewed By: zhanghang1989 Differential Revision: D36921926 fbshipit-source-id: e623b61633fb09fc25fa91d23555c62002045933
-
Yanghan Wang authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/280 T122435829 Reviewed By: zhanghang1989 Differential Revision: D36904298 fbshipit-source-id: 335c4cce199426b5f3b37cb2bfac04106e36ecd6
-
Facebook Community Bot authored
Co-authored-by:Facebook Community Bot <6422482+facebook-github-bot@users.noreply.github.com>
-
- 05 Jun, 2022 2 commits
-
-
Peizhao Zhang authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/277 Support cfg_diff for mismatched keys. Reviewed By: wat3rBro Differential Revision: D36737254 fbshipit-source-id: a1c189c92a24f3c109d9a427f135e53876b91624
-
Peizhao Zhang authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/259 Added randaug and trivialaug to d2go. * Only apply to images. Reviewed By: wat3rBro Differential Revision: D36605753 fbshipit-source-id: f72ea6f3de65ab115e4c6612343e93a092cbd7ea
-
- 03 Jun, 2022 3 commits
-
-
Zecheng He authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/269 Add deterministic instance option in RandomInstanceCrop. This feature can be useful for two-step model evaluation as every time we make the cropped instance deterministic. Reviewed By: wat3rBro Differential Revision: D36850404 fbshipit-source-id: ba6c0d9df0849e0110037f9d839b1c48f48096ad
-
snowytiger authored
-
Sam Tsai authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/273 Switching to PathManager which has more supported backends. Reviewed By: wat3rBro Differential Revision: D36731967 fbshipit-source-id: 672ea9bd2fd99d3c03dc144eff3d299306b1ce6e
-
- 02 Jun, 2022 2 commits
-
-
Miquel Jubert Hermoso authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/238 *This diff is part of a stack which has the goal of "buckifying" D2 (https://github.com/facebookresearch/d2go/commit/87374efb134e539090e0b5c476809dc35bf6aedb)Go core and enabling autodeps and other tooling. The last diff in the stack introduces the TARGETS. The diffs earlier in the stack are resolving circular dependencies and other issues which prevent the buckification from occurring.* Following the comments in an abandoned diff, split the export code into two files, which will have their corresponding dependencies: exporter and api. api.py contains the components which have little dependencies, so it can be imported basically anywhere without circular dependencies. exporter.py contains the utilities, which are use for export operations, for example in the exporter binary. Reviewed By: mcimpoi Differential Revision: D36166603 fbshipit-source-id: 25ded0b3925464c05be4048472a4c2ddcdb17ecf
-
Yanghan Wang authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/267 https://www.internalfb.com/intern/test/281475036363610?ref_report_id=0 is flaky, which is caused by running multiple tests at the same time, and clean up is not handled very well in that case. Reviewed By: tglik Differential Revision: D36787035 fbshipit-source-id: 6a478318fe011af936dd10fa564519c8c0615ed3
-
- 01 Jun, 2022 1 commit
-
-
Lawrence Chen authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/268 Adding a behavior in which the default D2 (https://github.com/facebookresearch/d2go/commit/87374efb134e539090e0b5c476809dc35bf6aedb)Go dataset mapper returns a dict that contains "transforms" stored in its original call. This enables dataset mapper subclasses to have access to the same transforms performed on the input. This behavior was originally present in the d2go_dataset_mapper_impl.py and d2go_dataset_mapper_impl_fb.py files, but was lost after the refactor to combine the files. Proposing to add this feature back in, to fix downstream bugs that relied on this behavior. Reviewed By: miqueljubert, wat3rBro Differential Revision: D36790367 fbshipit-source-id: 07e6e7bbfe9d764dccb430afe839df495af25fb7
-
- 31 May, 2022 1 commit
-
-
Yanghan Wang authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/266 CI has issue (eg. https://github.com/facebookresearch/d2go/runs/6662620169?check_suite_focus=true): ``` File "/usr/share/miniconda/envs/__setup_conda/lib/python3.8/site-packages/caffe2/proto/__init__.py", line 15, in <module> from caffe2.proto import caffe2_pb2, metanet_pb2, torch_pb2 File "/usr/share/miniconda/envs/__setup_conda/lib/python3.8/site-packages/caffe2/proto/caffe2_pb2.py", line 33, in <module> _descriptor.EnumValueDescriptor( File "/usr/share/miniconda/envs/__setup_conda/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 755, in __new__ _message.Message._CheckCalledFromGeneratedFile() TypeError: Descriptors cannot not be created directly. If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0. If you cannot immediately regenerate your protos, some other possible workarounds are: 1. Downgrade the protobuf package to 3.20.x or lower. 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower). More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates ``` It's caused by BC-breaking change of protobuf, fix the version. Reviewed By: tglik Differential Revision: D36781162 fbshipit-source-id: 8f13461a44a5a8f01abdc09a1ff8f9759fe55619
-
- 28 May, 2022 1 commit
-
-
Yanghan Wang authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/264 Reviewed By: tglik Differential Revision: D36427439 fbshipit-source-id: 3502d61ccb3c3f67d7ccfc1f55777c74f6c3b970
-
- 27 May, 2022 1 commit
-
-
Yanghan Wang authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/263 Re-use the code in D36651241 Reviewed By: tglik Differential Revision: D36682041 fbshipit-source-id: 1bf30afbbb69d10fc11f9161e29319f180c65e2f
-
- 26 May, 2022 1 commit
-
-
Yanghan Wang authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/262 synchronization should be handled inside creating data loader, otherwise we need to add synchronization everywhere besides the training-loop, which is impossible. Reviewed By: sstsai-adl Differential Revision: D36683362 fbshipit-source-id: 0bb7c9b50656fece5df6a007c37ec5888ee172bc
-
- 25 May, 2022 2 commits
-
-
Yanghan Wang authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/261 X-link: https://github.com/facebookresearch/mobile-vision/pull/71 `is_oss` and `fb_overwritable` are also needed in `mobile_cv`, move them from d2go. Reviewed By: zhanghang1989 Differential Revision: D36655821 fbshipit-source-id: 421c4d22d4c4620678908fe13d6e47ab39604ae7
-
Alan Li authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/260 Training + Evaluation for OCR Detection based on D2 (https://github.com/facebookresearch/d2go/commit/87374efb134e539090e0b5c476809dc35bf6aedb)GO Purpose: To restart training of OCR detection after abandonment. Updates filepaths and some configs, as well as addresses a race condition situation when creating temp json files for annotations read/write within D2 (https://github.com/facebookresearch/d2go/commit/87374efb134e539090e0b5c476809dc35bf6aedb)GO. The training can be restarted from scratch, or specified from a previous model by setting the weights_pth variable. Test plan is purely on 1 particular dataset. Reviewed By: sstsai-adl Differential Revision: D36632783 fbshipit-source-id: fe8677a57d660c495458d083c3fb70b95128b260
-
- 23 May, 2022 2 commits
-
-
Miquel Jubert Hermoso authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/239 *This diff is part of a stack which has the goal of "buckifying" D2 (https://github.com/facebookresearch/d2go/commit/87374efb134e539090e0b5c476809dc35bf6aedb)Go core and enabling autodeps and other tooling. The last diff in the stack introduces the TARGETS. The diffs earlier in the stack are resolving circular dependencies and other issues which prevent the buckification from occurring.* Break cyclic dependency by referring directly to the source file, instead to a different file that imports it. Reviewed By: tglik Differential Revision: D36166602 fbshipit-source-id: 7deafc02a52ab978a21593184d1b3d3810dc9346
-
Miquel Jubert Hermoso authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/225 *This diff is part of a stack which has the goal of "buckifying" D2 (https://github.com/facebookresearch/d2go/commit/87374efb134e539090e0b5c476809dc35bf6aedb)Go core and enabling autodeps and other tooling. The last diff in the stack introduces the TARGETS. The diffs earlier in the stack are resolving circular dependencies and other issues which prevent the buckification from occurring.* The overriding pattern applied in d2go_dataset_mapper, with the d2go_dataset_mapper_impl and d2go_dataset_mapper_impl_fb makes it possible that internal users get the _fb behaviour and external users the regular one, with the same import. But, this makes it necessary to put both files in the same dependency. This causes a circular dependency. In general, one reasonable assumption is that fb-only dependencies can import oss dependencies but not vice versa. In the current setup, grouping both _impl files in a buck target makes that buck target contain both fb and oss code, and depend on both. This is causing circular buck dependency issues. To fix this, while keeping the transparent import behavior, the following changes are done: 1. The implementation file is moved to a directory under .fb. It will have it's own target. 2. The non-fb version is renamed to DualInputDatasetMapper, as per Yanghan's suggestion. This simplifies the change, since it seems the fb version's behavior is not used atm. 3. d2go_dataset_mapper is moved to have the implementation itself Reviewed By: tglik Differential Revision: D35930993 fbshipit-source-id: ac57337d221df24f53e360d5dcb38ffa4164fef5
-
- 21 May, 2022 1 commit
-
-
Jerry Zhang authored
Summary: X-link: https://github.com/pytorch/pytorch/pull/77608 X-link: https://github.com/pytorch/fx2trt/pull/76 Pull Request resolved: https://github.com/facebookresearch/d2go/pull/249 X-link: https://github.com/fairinternal/ClassyVision/pull/104 X-link: https://github.com/pytorch/benchmark/pull/916 X-link: https://github.com/facebookresearch/ClassyVision/pull/791 X-link: https://github.com/facebookresearch/mobile-vision/pull/68 FX Graph Mode Quantization needs to know whether an fx node is a floating point Tensor before it can decide whether to insert observer/fake_quantize module or not, since we only insert observer/fake_quantize module for floating point Tensors. Currently we have some hacks to support this by defining some rules like NON_OBSERVABLE_ARG_DICT (https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/fx/utils.py#L496), but this approach is fragile and we do not plan to maintain it long term in the pytorch code base. As we discussed in the design review, we'd need to ask users to provide sample args and sample keyword args so that we can infer the type in a more robust way. This PR starts with changing the prepare_fx and prepare_qat_fx api to require user to either provide example arguments thrugh example_inputs, Note this api doesn't support kwargs, kwargs can make https://github.com/pytorch/pytorch/pull/76496#discussion_r861230047 (comment) simpler, but it will be rare, and even then we can still workaround with positional arguments, also torch.jit.trace(https://pytorch.org/docs/stable/generated/torch.jit.trace.html) and ShapeProp: https://github.com/pytorch/pytorch/blob/master/torch/fx/passes/shape_prop.py#L140 just have single positional args, we'll just use a single example_inputs argument for now. If needed, we can extend the api with an optional example_kwargs. e.g. in case when there are a lot of arguments for forward and it makes more sense to pass the arguments by keyword BC-breaking Note: Before: ```python m = resnet18(...) m = prepare_fx(m, qconfig_dict) # or m = prepare_qat_fx(m, qconfig_dict) ``` After: ```python m = resnet18(...) m = prepare_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),)) # or m = prepare_qat_fx(m, qconfig_dict, example_inputs=(torch.randn(1, 3, 224, 224),)) ``` Reviewed By: vkuzo, andrewor14 Differential Revision: D35984526 fbshipit-source-id: 706c8df71722c9aa5082a6491734f0144f0dd670
-
- 20 May, 2022 3 commits
-
-
Miquel Jubert Hermoso authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/233 *This diff is part of a stack which has the goal of "buckifying" D2 (https://github.com/facebookresearch/d2go/commit/87374efb134e539090e0b5c476809dc35bf6aedb)Go core and enabling autodeps and other tooling. The last diff in the stack introduces the TARGETS. The diffs earlier in the stack are resolving circular dependencies and other issues which prevent the buckification from occurring.* Again, a circular dependency between quantization/modeling -> meta_arch->rcnn. Solved in a similar way, they are caused by functions which only interact with the global registry. Separate the cause of the circular dependency into its own file, which will later have a separate TARGET. Reviewed By: wat3rBro Differential Revision: D35966052 fbshipit-source-id: c118a2af989e6c387641fe055a6734f9f0ab1db5
-
Miquel Jubert Hermoso authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/245 At the moment D2 (https://github.com/facebookresearch/d2go/commit/87374efb134e539090e0b5c476809dc35bf6aedb)Go's runner still uses the OSS pattern 1 (see wiki), where the files get remapped. This does not work with D2 (https://github.com/facebookresearch/d2go/commit/87374efb134e539090e0b5c476809dc35bf6aedb)Go, and makes it necessary to use some renaming tricks. This diff refactors the runner setup, to reduce the number of classes, and rely on fb_overwrite to add the correct fields to the config. Reviewed By: wat3rBro Differential Revision: D36316955 fbshipit-source-id: 4aaaece121b8df802f9395648c97a647fa7db857
-
Yanghao Li authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/258 Support square padding case in backbone. Reviewed By: wat3rBro Differential Revision: D35552076 fbshipit-source-id: e4f7f4da62b6ee9b71686071ff6cf2747ecc90e0
-
- 19 May, 2022 5 commits
-
-
Miquel Jubert Hermoso authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/254 WTTS Reviewed By: wat3rBro Differential Revision: D36438060 fbshipit-source-id: 440daa5d238e7902606b3709ec7887260f1a20ec
-
Miquel Jubert Hermoso authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/256 WTTS Reviewed By: wat3rBro Differential Revision: D36438057 fbshipit-source-id: 4d9bfda43d0263e3271b69172bf02f8f0b5d2663
-
Miquel Jubert Hermoso authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/257 WTTS Reviewed By: wat3rBro Differential Revision: D36438059 fbshipit-source-id: 010cfef86795dc65ed279124c1ed32206829fdaa
-
Miquel Jubert Hermoso authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/253 WTTS Reviewed By: wat3rBro Differential Revision: D36438058 fbshipit-source-id: 1909cd31481eca92474baf952bd227256c786ac2
-
Miquel Jubert Hermoso authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/255 WTTS Reviewed By: wat3rBro Differential Revision: D36438007 fbshipit-source-id: 25b53b3ed1e048d48ab1085b5661bb47cc085c55
-
- 18 May, 2022 1 commit
-
-
Bo (YANG, Bo) Yang authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/251 As title Reviewed By: wenliangzhao2018 Differential Revision: D36360786 fbshipit-source-id: c52fa9a108a908fb96d16b2affd18bed19efff5e
-
- 17 May, 2022 2 commits
-
-
Peizhao Zhang authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/208 Support unapplying modeling hooks. Reviewed By: tglik Differential Revision: D35540649 fbshipit-source-id: 60cc5e214282e30b39fc98ba4d58dad2fc6ea086
-
Peizhao Zhang authored
Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/207 Apply modeling hook when building the model from cfg. Differential Revision: D35535571 fbshipit-source-id: e80dd3912911e49c6ed60477f3ba52f74a220dec
-