Skip to content

GitLab

"server/vscode:/vscode.git/clone" did not exist on "b2554455572b28c0e18423d6fe6896cf7137dbd6"
Switch branch/tag
  1. 30 Jun, 2022 1 commit
  3. 29 Jun, 2022 1 commit
  5. 24 Jun, 2022 1 commit
    • Yanghan Wang's avatar
      use runner class instead of instance outside of main · 8051775c
      Yanghan Wang authored 
      Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/312

As discussed, we decided to not use runner instance outside of `main`, previous diffs already solved the prerequisites, this diff mainly does the renaming.
- Use runner name (str) in the fblearner, ML pipeline.
- Use runner name (str) in FBL operator, MAST and binary operator.
- Use runner class as the interface of main, it can be either the name of class (str) or actual class. The main usage should be using `str`, so that the importing of class happens inside `main`. But it's also a common use case to import runner class and call `main` for things like ad-hoc scripts or tests, supporting actual class makes it easier modify code for those cases (eg. some local test class doesn't have a name, so it's not feasible to use runner name).

Reviewed By: newstzpz

Differential Revision: D37060338

fbshipit-source-id: 879852d41902b87d6db6cb9d7b3e8dc55dc4b976
      8051775c
  7. 16 Jun, 2022 1 commit
  9. 14 Jun, 2022 1 commit
  11. 09 Jun, 2022 1 commit
  13. 15 May, 2022 1 commit
    • John Reese's avatar
      apply import merging for fbcode (7 of 11) · b3a9204c
      John Reese authored 
      Summary:
Applies new import merging and sorting from µsort v1.0.

When merging imports, µsort will make a best-effort to move associated
comments to match merged elements, but there are known limitations due to
the diynamic nature of Python and developer tooling. These changes should
not produce any dangerous runtime changes, but may require touch-ups to
satisfy linters and other tooling.

Note that µsort uses case-insensitive, lexicographical sorting, which
results in a different ordering compared to isort. This provides a more
consistent sorting order, matching the case-insensitive order used when
sorting import statements by module name, and ensures that "frog", "FROG",
and "Frog" always sort next to each other.

For details on µsort's sorting and merging semantics, see the user guide:
https://usort.readthedocs.io/en/stable/guide.html#sorting

Reviewed By: lisroach

Differential Revision: D36402205

fbshipit-source-id: a4efc688d02da80c6e96685aa8eb00411615a366
      b3a9204c
  15. 14 May, 2022 1 commit
  17. 10 Mar, 2022 1 commit
  19. 08 Jan, 2022 1 commit
    • Binh Tang's avatar
      Add deprecation path for renamed training type plugins (#11227) · fcd51171
      Binh Tang authored 
      Summary:
### New commit log messages
  4eede7c30 Add deprecation path for renamed training type plugins (#11227)

Reviewed By: edward-io, daniellepintz

Differential Revision: D33409991

fbshipit-source-id: 373e48767e992d67db3c85e436648481ad16c9d0
      fcd51171
  21. 06 Jan, 2022 1 commit
    • Binh Tang's avatar
      Rename `DDPPlugin` to `DDPStrategy` (#11142) · aeb15613
      Binh Tang authored 
      Summary:
### New commit log messages
  b64dea9dc Rename `DDPPlugin` to `DDPStrategy` (#11142)

Reviewed By: jjenniferdai

Differential Revision: D33259306

fbshipit-source-id: b4608c6b96b4a7977eaa4ed3f03c4b824882aef0
      aeb15613
  23. 29 Dec, 2021 1 commit
  25. 24 Sep, 2021 1 commit
  27. 09 Sep, 2021 1 commit
  29. 07 Jul, 2021 1 commit
  31. 07 Jun, 2021 1 commit
  33. 25 May, 2021 2 commits
  35. 13 May, 2021 1 commit
    • Kai Zhang's avatar
      Auto scale config for multi-node training · e87ed5f0
      Kai Zhang authored 
      Summary:
Pull Request resolved: https://github.com/facebookresearch/d2go/pull/62

Lightning trainer set max step to cfg.SOLVER.MAX_ITER. However, this is the max iteration for all nodes, in multi-node training, we need to scale it down, as well as eval period and other configs.
This diff calls `auto_scale_world_size` before passing the config to trainer.

Reviewed By: wat3rBro

Differential Revision: D28140877

fbshipit-source-id: 2639ae58773a4ec2a0cc59dfefb2f5d9b1afe1a8
      e87ed5f0
  37. 17 Apr, 2021 2 commits
    • Kai Zhang's avatar
      Delegate to model's customization · aeb24a92
      Kai Zhang authored 
      Summary: Delegate FX quantization callback's customization to model.

Reviewed By: wat3rBro

Differential Revision: D27669212

fbshipit-source-id: 2715546cf03134896da6f95ecddaf8503ff95d0b
      aeb24a92
    • Kai Zhang's avatar
      E2E QAT Workflow on Lightning · 845d0b2c
      Kai Zhang authored 
      Summary:
As per title and sanity test E2E QAT workflow on Lightning Trainer.

- add `post_training_opts`. This is required to use `all_steps_qat.json` with Lightning. We don't actually support the post_training_opts in this diff though - we leave it part of T83437359.
- Update .yaml to specify the Quantize-able modules.
- Update `lightning_train_net.py` to use the QuantizationAwareTraining callback.

Reviewed By: kandluis

Differential Revision: D26304879

fbshipit-source-id: 948bef4817d385d8a0969e4990d7f17ecd6994b7
      845d0b2c
  39. 09 Apr, 2021 1 commit
    • Ananth Subramaniam's avatar
      Make checkpointing tests slightly less restrictive · fc5616c8
      Ananth Subramaniam authored 
      Summary:
Before: this test would assume only 2 checkpoints were stored: `last.ckpt`, and `FINAL_MODEL_CKPT`
Now: this test asserts that at least these 2 checkpoints are stored. In case the config specifies `save_top_k=-1` for instance, we'd save more checkpoints, causing this test to fail

Since this test is only loading the last and the final outputs, I'm changing the behavior to assert that these checkpoints must be saved and ignoring other checkpoint files that could be generated.

Reviewed By: kazhang

Differential Revision: D27671284

fbshipit-source-id: 0419fb46856d048e7b6eba3ff1dc65b7280a9a90
      fc5616c8
  41. 24 Mar, 2021 2 commits
    • Kai Zhang's avatar
      Support evaluate predictor · 6aec097e
      Kai Zhang authored 
      Summary:
Evaluate the predictor generated by previous step.
This diff modify the lightning_train_net to reuse the evaluation logic by adding a `predictor_path` param.
This diff also makes Lightning training backend depends on `cfg.MODEL.DEVICE` so that in evaluate_predictor step, user could set backend by changing model device. This is useful for evaluating int8 quantized model.

Reviewed By: newstzpz

Differential Revision: D27150609

fbshipit-source-id: fb72da3e81db932c0fa479350150720143e09a3e
      6aec097e
    • Kai Zhang's avatar
      Simplify Lightning task and model creation · 9051f71a
      Kai Zhang authored 
      Summary:
Given that the way to create D2 (https://github.com/facebookresearch/d2go/commit/465cdb842513eb910aa20fcedea1d2edd15dc7b7)go runner and Lightning task are different, get_class was introduced so that in application we could do:
```
if is Lightning:
    task_cls = get_class(classname)
    task = task_cls(cfg)
else:
    runner = create_runner(classname)
```
It turns out that we could need to do that in many places: workflow, binaries.
This diff revert `get_class` and return class in `create_runner` if the class is a Lightning module.

Reviewed By: newstzpz

Differential Revision: D26676595

fbshipit-source-id: c3ce2016d09fe073af4c2dd9f98eea4e59ca621b
      9051f71a
  43. 03 Mar, 2021 2 commits
    • Kai Zhang's avatar
      Split lightning_train_net into OSS and internal · 857195d8
      Kai Zhang authored 
      Summary:
As titled. The OSS version only use PyTorch Lightning while internal version leverages some features(e.g. Manifold integration, every_n_step checkpointing).
This diff splits train_net.main into smaller functions so that they could be shared across OSS and internal versions.

Reviewed By: zhanghang1989

Differential Revision: D26752701

fbshipit-source-id: 7f68e2a81e78193e117517a0ff668ab14b76ea65
      857195d8
    • facebook-github-bot's avatar
      Initial commit · f23248c0
      facebook-github-bot authored 
      fbshipit-source-id: f4a8ba78691d8cf46e003ef0bd2e95f170932778
      f23248c0