GitLab

Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/388

Reviewed By: wat3rBro

Differential Revision: D40377653

fbshipit-source-id: 3f99d30480a801c794665e67bb2b0d28c7c5b0e5

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/401

as followup on D40001329 (https://github.com/facebookresearch/d2go/commit/69bf820c64cd0ffb6a84f465199c9134814cf58e).
The export is running main func without launching distributed workers, so it need to set the shared context explicitly.

Reviewed By: wat3rBro

Differential Revision: D40708631

fbshipit-source-id: 7689a45dff383ba2cce01d33d3be95d612269fbe

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/356

Attaching PDB on failure is not working when running in distributed environment. This change allows to disable this behavior by passing a command line argument.

Reviewed By: miqueljubert

Differential Revision: D38514736

fbshipit-source-id: 2e0008d6fbc6a4518a605debe67d76f8354364fc

Summary: Pull Request resolved: https://github.com/facebookresearch/d2go/pull/340

Reviewed By: miqueljubert

Differential Revision: D37968017

fbshipit-source-id: a3953fdbb2c48ceaffcf94df081c0b3253d247d5

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/312

As discussed, we decided to not use runner instance outside of `main`, previous diffs already solved the prerequisites, this diff mainly does the renaming.
- Use runner name (str) in the fblearner, ML pipeline.
- Use runner name (str) in FBL operator, MAST and binary operator.
- Use runner class as the interface of main, it can be either the name of class (str) or actual class. The main usage should be using `str`, so that the importing of class happens inside `main`. But it's also a common use case to import runner class and call `main` for things like ad-hoc scripts or tests, supporting actual class makes it easier modify code for those cases (eg. some local test class doesn't have a name, so it's not feasible to use runner name).

Reviewed By: newstzpz

Differential Revision: D37060338

fbshipit-source-id: 879852d41902b87d6db6cb9d7b3e8dc55dc4b976

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/302

Fixing issue introduced in D35035813 (https://github.com/facebookresearch/d2go/commit/744d72d73b7103b8dd9ca69372a179b44ad7d733) that break the OSS cli tools defined in https://github.com/facebookresearch/d2go/blob/8098d160c0b38b796a2c164719650a50238a0f89/setup.py#L87-L92.
The cli alias in setup need a function without any args to call. So creating a new main_cli function

Reviewed By: wat3rBro

Differential Revision: D37210948

fbshipit-source-id: efb3df15e9933c617414a727e5b53553db170622

Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/238

*This diff is part of a stack which has the goal of "buckifying" D2 (https://github.com/facebookresearch/d2go/commit/87374efb134e539090e0b5c476809dc35bf6aedb)Go core and enabling autodeps and other tooling. The last diff in the stack introduces the TARGETS. The diffs earlier in the stack are resolving circular dependencies and other issues which prevent the buckification from occurring.*

Following the comments in an abandoned diff, split the export code into two files, which will have their corresponding dependencies: exporter and api. api.py contains the components which have little dependencies, so it can be imported basically anywhere without circular dependencies.

exporter.py contains the utilities, which are use for export operations, for example in the exporter binary.

Reviewed By: mcimpoi

Differential Revision: D36166603

fbshipit-source-id: 25ded0b3925464c05be4048472a4c2ddcdb17ecf

Summary:
Applies new import merging and sorting from µsort v1.0.

When merging imports, µsort will make a best-effort to move associated
comments to match merged elements, but there are known limitations due to
the diynamic nature of Python and developer tooling. These changes should
not produce any dangerous runtime changes, but may require touch-ups to
satisfy linters and other tooling.

Note that µsort uses case-insensitive, lexicographical sorting, which
results in a different ordering compared to isort. This provides a more
consistent sorting order, matching the case-insensitive order used when
sorting import statements by module name, and ensures that "frog", "FROG",
and "Frog" always sort next to each other.

For details on µsort's sorting and merging semantics, see the user guide:
https://usort.readthedocs.io/en/stable/guide.html#sorting

Reviewed By: lisroach

Differential Revision: D36402205

fbshipit-source-id: a4efc688d02da80c6e96685aa8eb00411615a366

Summary: Tweak exporter and evaluator cli entry point func to support calling it as a module with args from custom launching code.

Reviewed By: sstsai-adl

Differential Revision: D35035813

fbshipit-source-id: c8b24099e94ccc58c184f8aac95b2a24a137e86a

Summary:
Currently, the exporter method takes in a compare_accuracy parameter, which after all the compute (exporting etc.) raises an exception if it is set to True.

This looks like an antipattern, and causes a waste of compute.

Therefore, I am proposing to raise the exception at the very beginning of method call to let the client know in advance that this argument's functionality isn't implemented yet.

NOTE: We might also choose to get rid of the entire parameter. I am open for suggestions.

Differential Revision: D34186578

fbshipit-source-id: d7fbe7589dfe2d2f688b870885ca61e6829c9329

Summary: make it an option

Differential Revision: D32601981

fbshipit-source-id: 308a0c49939531d840914aa8e256aae6db463929

Differential Revision: D30973518

fbshipit-source-id: fbdfb862ab23d5141553499471f92d2218addf91

Summary:
Added predictor_type `boltnn_int8` to export to BoltNN via torch delegate.

- `int8` needs to be in the name, otherwise the post-train quantization won't happen;

```
cfg.QUANTIZATION.BACKEND = "qnnpack"
// cfg.QUANTIZATION.CUSTOM_QSCHEME = "per_tensor_affine"
```

Seems that ` QUANTIZATION.CUSTOM_QSCHEME per_tensor_affine` is not needed - likely covered by "qnnpack".

Reviewed By: wat3rBro

Differential Revision: D29106043

fbshipit-source-id: 865ac5af86919fe7b4530b48433a1bd11e295bf4

Summary: Use all training dataset for export instead of just first. This is to support use cases where there is only a small amount of images per jsons but a number of jsons. Since calibration uses the first dataset, it is limited by the number of images in a single dataset.

Reviewed By: ppwwyyxx

Differential Revision: D28902673

fbshipit-source-id: f80146b02d2d1bc04703fbb21ef410f5e26ba64c

Differential Revision:
D27881742 (https://github.com/facebookresearch/d2go/commit/90aff5daf608473dd312b300db8615326fa40a37)

Original commit changeset: 34a3ab7a88f4

fbshipit-source-id: 42c03b4f2b69c656b26774a4665b84b832262650

Summary:
- Enable sdk inference config specification in export step. This enables adding the sdk configuration as part of model file in the export step. The sdk config can be specified as infernece_config.yaml and is zipped together with torchscript model. The main goal of sdk configuration is to control the model inference behavior with model.
- SDK inference config design doc: https://docs.google.com/document/d/1j5qx8IrnFg1DJFzTnu4W8WmXFYJ-AgCDfSQHb2ACJsk/edit
- One click fblearner pipeline is in next diff on the stack

Differential Revision: D27881742

fbshipit-source-id: 34a3ab7a88f456b74841cf671ea1b3f678cdb733

Summary:
Fix typos in exporter

Pull Request resolved: https://github.com/facebookresearch/d2go/pull/45

Reviewed By: wat3rBro

Differential Revision: D27779963

Pulled By: zhanghang1989

fbshipit-source-id: bcf7922afe6d4cccc074615069538eb5a6098b98

Summary:
Change depoyment to deployment in README.md.
Change datasest to datasets in tools/exporter.py.

Pull Request resolved: https://github.com/facebookresearch/d2go/pull/7

Reviewed By: newstzpz

Differential Revision: D26821039

Pulled By: zhanghang1989

fbshipit-source-id: 5056d15c877c4b3d771d33267139e73f1527da21

fbshipit-source-id: f4a8ba78691d8cf46e003ef0bd2e95f170932778