GitLab

Summary: We now expect pytorch 1.12

Reviewed By: patricklabatut

Differential Revision: D38425758

fbshipit-source-id: a22e672fcb0dc18f7d6424323d9cc6aaf5fcb4c6

Summary:
LLFF (and most/all non-synth datasets) will have no background/foreground distinction. Add support for data with no fg mask.

Also, we had a bug in stats loading, like this:
  * Load stats
  * One of the stats has a history of length 0
  * That's fine, e.g. maybe it's fg_error but the dataset has no notion of fg/bg. So leave it as len 0
  * Check whether all the stats have the same history length as an arbitrarily chosen "reference-stat"
  * Ooops the reference-stat happened to be the stat with length 0
  * assert (legit_stat_len == reference_stat_len (=0)) ---> failed assert

Also some minor fixes (from Jeremy's other diff) to support LLFF

Reviewed By: davnov134

Differential Revision: D38475272

fbshipit-source-id: 5b35ac86d1d5239759f537621f41a3aa4eb3bd68

Differential Revision: D38529199

fbshipit-source-id: 4bc0574493b60f13e08d9ea7bded862778b3d171

Summary: Adding the implicitron tutorials to the website

Reviewed By: kjchalup

Differential Revision: D38508673

fbshipit-source-id: b38d43174acf6e1355b14bf6d58513f6696e7fa0

Summary: In a multisequence (fewview) scenario, it does not make sense to use all cameras for evaluating the difficulty as they come from different scenes. Using only this batch’s source (known) cameras instead.

Reviewed By: bottler

Differential Revision: D38491070

fbshipit-source-id: d6312d8fbb125b28a33db9f53d4215bcd1ca28a8

Summary:
Request in https://github.com/facebookresearch/pytorch3d/issues/1233 for option to disable CUDA build.

Also option to disable binary build completely. This could be useful e.g. in the config tutorial where we need a small python-only part of pytorch3d.

Reviewed By: kjchalup

Differential Revision: D38458624

fbshipit-source-id: 421a0b1cc31306d7e322d3e743e30a7533d7f034

Summary:
simply fixing a typo in _ls function

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1281

Reviewed By: patricklabatut

Differential Revision: D38457259

Pulled By: bottler

fbshipit-source-id: 5204a57cb4d1fe1c804d8af3301b8ea2945443e8

Summary:
Misc fixes.

- most important: the mac image is gone so switch to a newer one.
- torch.concat is new; was used accidentally
- remove lpips from testing in meta.yaml as it is breaking the conda test. Better to leave the relevant tests failing in OSS.
- TypedDict usage is breaking implicitron on Python 3.7.

Reviewed By: patricklabatut

Differential Revision: D38458164

fbshipit-source-id: b16c26453a743b9a771e2a6787b9a4d2a52e41c2

Summary: This field is specific to one purpose.

Reviewed By: patricklabatut

Differential Revision: D38424891

fbshipit-source-id: e017304497012430c30e436da7052b9ad6fc7614

Summary: remove n_instances==0 special case, standardise args for GlobalEncoderBase's forward.

Reviewed By: shapovalov

Differential Revision: D37817340

fbshipit-source-id: 0aac5fbc7c336d09be9d412cffff5712bda27290

Summary: One way to tidy the installation so we don't install files in site-packages/projects. Fixes https://github.com/facebookresearch/pytorch3d/issues/1279

Reviewed By: shapovalov, davnov134

Differential Revision: D38426772

fbshipit-source-id: ac1a54fbf230adb53904701e1f38bf9567f647ce

Differential Revision: D38448021

fbshipit-source-id: 966708035115b4870a74f0ca0bbf8ea88b853527

Summary: new

Reviewed By: kjchalup

Differential Revision: D38425731

fbshipit-source-id: 0fd8f524df6b29ceb8c7c9a674022412c1efc3b5

Summary: new

Reviewed By: kjchalup

Differential Revision: D38425739

fbshipit-source-id: 23e7a1dd01c034b231511acecf40ea0deb9a067f

Summary: ... see title

Reviewed By: bottler

Differential Revision: D38430103

fbshipit-source-id: fd0696da82c367b0f4d9c9d92c0cba0230834137

Summary: continued - avoid duplicate inputs

Reviewed By: davnov134

Differential Revision: D38248827

fbshipit-source-id: 91ed398e304496a936f66e7a70ab3d189eeb5c70

Summary: continued - don't duplicate inputs

Reviewed By: kjchalup

Differential Revision: D38248829

fbshipit-source-id: 2d56418ecbec9cc597c3cf0c122199e274661516

Summary: Don't copy from one part of config to another, rather do the copy within GenericModel.

Reviewed By: davnov134

Differential Revision: D38248828

fbshipit-source-id: ff8af985c37ea1f7df9e0aa0a45a58df34c3f893

Summary: Made the config system call open_dict when it calls the tweak function.

Reviewed By: shapovalov

Differential Revision: D38315334

fbshipit-source-id: 5924a92d8d0bf399bbf3788247f81fc990e265e7

Summary:
Stats are logically connected to the training loop, not to the model. Hence, moving to the training loop.

Also removing resume_epoch from OptimizerFactory in favor of a single place - ModelFactory. This removes the need for config consistency checks etc.

Reviewed By: kjchalup

Differential Revision: D38313475

fbshipit-source-id: a1d188a63e28459df381ff98ad8acdcdb14887b7

Summary: Blender data doesn't have depths or crops.

Reviewed By: shapovalov

Differential Revision: D38345583

fbshipit-source-id: a19300daf666bbfd799d0038aeefa14641c559d7

Summary: Simple DataLoaderMapProvider instance

Reviewed By: davnov134

Differential Revision: D38326719

fbshipit-source-id: 58556833e76fae5790d25a59bea0aac4ce046bf1

Summary: fix to the D38275943 (https://github.com/facebookresearch/pytorch3d/commit/597e0259dc43bf4903e9c99f5d61410c1ad75b78).

Reviewed By: bottler

Differential Revision: D38355683

fbshipit-source-id: f326f45279fafa57f24b9211ebd3fda18a518937

Summary: Before this diff, train_stats.py would not be created by default, EXCEPT when resuming training. This makes them appear from start.

Reviewed By: shapovalov

Differential Revision: D38320341

fbshipit-source-id: 8ea5b99ec81c377ae129f58e78dc2eaff94821ad

Summary: Remove the dataset's need to provide the task type.

Reviewed By: davnov134, kjchalup

Differential Revision: D38314000

fbshipit-source-id: 3805d885b5d4528abdc78c0da03247edb9abf3f7

Summary:
Added _NEED_CONTROL
 to JsonIndexDatasetMapProviderV2 and made dataset_tweak_args use it.

Reviewed By: bottler

Differential Revision: D38313914

fbshipit-source-id: 529847571065dfba995b609a66737bd91e002cfe

Summary: Only import it if you ask for it.

Reviewed By: kjchalup

Differential Revision: D38327167

fbshipit-source-id: 3f05231f26eda582a63afc71b669996342b0c6f9

Summary: <see title>

Reviewed By: bottler

Differential Revision: D38314727

fbshipit-source-id: 7178b816a22b06af938a35c5f7bb88404fb1b1c4

Summary: Made eval_batches be set in call to `__init__` not after the construction as they were before

Reviewed By: bottler

Differential Revision: D38275943

fbshipit-source-id: 32737401d1ddd16c284e1851b7a91f8b041c406f

Summary: Currently, seeds are set only inside the train loop. But this does not ensure that the model weights are initialized the same way everywhere which makes all experiments irreproducible. This diff fixes it.

Reviewed By: bottler

Differential Revision: D38315840

fbshipit-source-id: 3d2ecebbc36072c2b68dd3cd8c5e30708e7dd808

Summary: Fixes the MC rasterization bug

Reviewed By: bottler

Differential Revision: D38312234

fbshipit-source-id: 910cf809ef3faff3de7a8d905b0821f395a52edf

Summary: Make a dummy single-scene dataset using the code from generate_cow_renders (used in existing NeRF tutorials)

Reviewed By: kjchalup

Differential Revision: D38116910

fbshipit-source-id: 8db6df7098aa221c81d392e5cd21b0e67f65bd70

Summary:
This large diff rewrites a significant portion of Implicitron's config hierarchy. The new hierarchy, and some of the default implementation classes, are as follows:
```
Experiment
    data_source: ImplicitronDataSource
        dataset_map_provider
        data_loader_map_provider
    model_factory: ImplicitronModelFactory
        model: GenericModel
    optimizer_factory: ImplicitronOptimizerFactory
    training_loop: ImplicitronTrainingLoop
        evaluator: ImplicitronEvaluator
```

1) Experiment (used to be ExperimentConfig) is now a top-level Configurable and contains as members mainly (mostly new) high-level factory Configurables.
2) Experiment's job is to run factories, do some accelerate setup and then pass the results to the main training loop.
3) ImplicitronOptimizerFactory and ImplicitronModelFactory are new high-level factories that create the optimizer, scheduler, model, and stats objects.
4) TrainingLoop is a new configurable that runs the main training loop and the inner train-validate step.
5) Evaluator is a new configurable that TrainingLoop uses to run validation/test steps.
6) GenericModel is not the only model choice anymore. Instead, ImplicitronModelBase (by default instantiated with GenericModel) is a member of Experiment and can be easily replaced by a custom implementation by the user.

All the new Configurables are children of ReplaceableBase, and can be easily replaced with custom implementations.

In addition, I added support for the exponential LR schedule, updated the config files and the test, as well as added a config file that reproduces NERF results and a test to run the repro experiment.

Reviewed By: bottler

Differential Revision: D37723227

fbshipit-source-id: b36bee880d6aa53efdd2abfaae4489d8ab1e8a27

Summary:
This is an internal change in the config systen. It allows redefining a pluggable implementation with new default values. This is useful in notebooks / interactive use. For example, this now works.

        class A(ReplaceableBase):
            pass

        registry.register
        class B(A):
            i: int = 4

        class C(Configurable):
            a: A
            a_class_type: str = "B"

            def __post_init__(self):
                run_auto_creation(self)

        expand_args_fields(C)

        registry.register
        class B(A):
            i: int = 5

        c = C()

        assert c.a.i == 5

Reviewed By: shapovalov

Differential Revision: D38219371

fbshipit-source-id: 72911a9bd3426d3359cf8802cc016fc7f6d7713b

Summary:
Adding MeshRasterizerOpenGL, a faster alternative to MeshRasterizer. The new rasterizer follows the ideas from "Differentiable Surface Rendering via non-Differentiable Sampling".

The new rasterizer 20x faster on a 2M face mesh (try pose optimization on Nefertiti from https://www.cs.cmu.edu/~kmcrane/Projects/ModelRepository/!). The larger the mesh, the larger the speedup.

There are two main disadvantages:
* The new rasterizer works with an OpenGL backend, so requires pycuda.gl and pyopengl installed (though we avoided writing any C++ code, everything is in Python!)
* The new rasterizer is non-differentiable. However, you can still differentiate the rendering function if you use if with the new SplatterPhongShader which we recently added to PyTorch3D (see the original paper cited above).

Reviewed By: patricklabatut, jcjohnson

Differential Revision: D37698816

fbshipit-source-id: 54d120639d3cb001f096237807e54aced0acda25

Summary: Needed to properly change devices during OpenGL rasterization.

Reviewed By: jcjohnson

Differential Revision: D37698568

fbshipit-source-id: 38968149d577322e662d3b5d04880204b0a7be29

Summary:
EGLContext is a utility to render with OpenGL without an attached display (that is, without a monitor).

DeviceContextManager allows us to avoid unnecessary context creations and releases. See docstrings for more info.

Reviewed By: jcjohnson

Differential Revision: D36562551

fbshipit-source-id: eb0d2a2f85555ee110e203d435a44ad243281d2c

Summary: Error when sending an unbatched FrameData through GM.

Reviewed By: shapovalov

Differential Revision: D38036286

fbshipit-source-id: b8d280c61fbbefdc112c57ccd630ab3ccce7b44e

Summary: Avoid calculating all_train_cameras before it is needed, because it is slow in some datasets.

Reviewed By: shapovalov

Differential Revision: D38037157

fbshipit-source-id: 95461226655cde2626b680661951ab17ebb0ec75

Summary: lint issues (mostly flake) in implicitron

Reviewed By: patricklabatut

Differential Revision: D37920948

fbshipit-source-id: 8cb3c2a2838d111c80a211c98a404c210d4649ed