GitLab

Summary: The ImplicitronDataset class corresponds to JsonIndexDatasetMapProvider

Reviewed By: shapovalov

Differential Revision: D36661396

fbshipit-source-id: 80ca2ff81ef9ecc2e3d1f4e1cd14b6f66a7ec34d

Summary: replace dataloader_zoo with a pluggable DataLoaderMapProvider.

Reviewed By: shapovalov

Differential Revision: D36475441

fbshipit-source-id: d16abb190d876940434329928f2e3f2794a25416

Summary: replace dataset_zoo with a pluggable DatasetMapProvider. The logic is now in annotated_file_dataset_map_provider.

Reviewed By: shapovalov

Differential Revision: D36443965

fbshipit-source-id: 9087649802810055e150b2fbfcc3c197a761f28a

Summary: Separate ImplicitronDatasetBase and FrameData (to be used by all data sources) from ImplicitronDataset (which is specific).

Reviewed By: shapovalov

Differential Revision: D36413111

fbshipit-source-id: 3725744cde2e08baa11aff4048237ba10c7efbc6

Summary:
Move dataset_args and dataloader_args from ExperimentConfig into a new member called datasource so that it can contain replaceables.

Also add enum Task for task type.

Reviewed By: shapovalov

Differential Revision: D36201719

fbshipit-source-id: 47d6967bfea3b7b146b6bbd1572e0457c9365871

Summary: Make ResNetFeatureExtractor be an implementation of FeatureExtractorBase.

Reviewed By: davnov134

Differential Revision: D35433098

fbshipit-source-id: 0664a9166a88e150231cfe2eceba017ae55aed3a

Summary:
1. Typo in the dataset path in the config.
2. Typo in num_frames.
3. Pick sequence was cached before it was modified for single-sequence.

Reviewed By: bottler

Differential Revision: D36417329

fbshipit-source-id: 6dcd75583de510412e1ae58f63db04bb4447403e

Summary: Stronger typing for these functions

Reviewed By: shapovalov

Differential Revision: D36170489

fbshipit-source-id: a2104b29dbbbcfcf91ae1d076cd6b0e3d2030c0b

Summary:
This converts raysamplers to ReplaceableBase so that users can hack their own raysampling impls.

Context: Andrea tried to implement TensoRF within implicitron but could not due to the need to implement his own raysampler.

Reviewed By: shapovalov

Differential Revision: D36016318

fbshipit-source-id: ef746f3365282bdfa9c15f7b371090a5aae7f8da

Summary: Implements a ViewPooler that groups ViewSampler and FeatureAggregator.

Reviewed By: shapovalov

Differential Revision: D35852367

fbshipit-source-id: c1bcaf5a1f826ff94efce53aa5836121ad9c50ec

Summary:
Applies the black-fbsource codemod with the new build of pyfmt.

paintitblack

Reviewed By: lisroach

Differential Revision: D36324783

fbshipit-source-id: 280c09e88257e5e569ab729691165d8dedd767bc

Summary:
To avoid model_zoo, we need to make GenericModel pluggable.
I also align creation APIs for convenience.

Reviewed By: bottler, davnov134

Differential Revision: D35933093

fbshipit-source-id: 8228926528eb41a795fbfbe32304b8019197e2b1

Summary:
Applies new import merging and sorting from µsort v1.0.

When merging imports, µsort will make a best-effort to move associated
comments to match merged elements, but there are known limitations due to
the diynamic nature of Python and developer tooling. These changes should
not produce any dangerous runtime changes, but may require touch-ups to
satisfy linters and other tooling.

Note that µsort uses case-insensitive, lexicographical sorting, which
results in a different ordering compared to isort. This provides a more
consistent sorting order, matching the case-insensitive order used when
sorting import statements by module name, and ensures that "frog", "FROG",
and "Frog" always sort next to each other.

For details on µsort's sorting and merging semantics, see the user guide:
https://usort.readthedocs.io/en/stable/guide.html#sorting

Reviewed By: bottler

Differential Revision: D35553814

fbshipit-source-id: be49bdb6a4c25264ff8d4db3a601f18736d17be1

Summary:
Try again to solve https://github.com/facebookresearch/pytorch3d/issues/1144 pickling problem.
D35258561 (https://github.com/facebookresearch/pytorch3d/commit/24260130ce382640a89e978cd671298488ec6451) didn't work.

When writing a function or vanilla class C which you want people to be able to call get_default_args on, you must add the line enable_get_default_args(C) to it. This causes autogeneration of a hidden dataclass in the module.

Reviewed By: davnov134

Differential Revision: D35364410

fbshipit-source-id: 53f6e6fff43e7142ae18ca3b06de7d0c849ef965

Summary:
ListConfig and DictConfig members of get_default_args(X) when X is a callable will contain references to a temporary dataclass and therefore be unpicklable. Avoid this in a few cases.

Fixes https://github.com/facebookresearch/pytorch3d/issues/1144

Reviewed By: shapovalov

Differential Revision: D35258561

fbshipit-source-id: e52186825f52accee9a899e466967a4ff71b3d25

Summary: Enable `pytorch3d_implicitron_runner` executable

Reviewed By: shapovalov

Differential Revision: D34754902

fbshipit-source-id: 213f3e9183e3f7dd7b4df16ad77d95fbc971d625

Summary:
Before the fix, running get_default_args(C: Callable) returns an unstructured DictConfig which causes Enums to be handled incorrectly. This is a fix.

WIP update: Currently tests still fail whenever a function signature contains an untyped argument: This needs to be somehow fixed.

Reviewed By: bottler

Differential Revision: D34932124

fbshipit-source-id: ecdc45c738633cfea5caa7480ba4f790ece931e8

Summary: Using the API from D35012121 everywhere.

Reviewed By: bottler

Differential Revision: D35045870

fbshipit-source-id: dab112b5e04160334859bbe8fa2366344b6e0f70

Co-authored-by: Jeremy Francis Reizenstein <bottler@users.noreply.github.com>

Summary: Move this simple layer from the NeRF project into pytorch3d.

Reviewed By: shapovalov

Differential Revision: D34126972

fbshipit-source-id: a9c6d6c3c1b662c1b844ea5d1b982007d4df83e6

Summary: Migrate away from NDCGridRaysampler and GridRaysampler to their more flexible replacements.

Reviewed By: patricklabatut

Differential Revision: D33281584

fbshipit-source-id: 65f8702e700a32d38f7cd6bda3924bb1707a0633

Summary: New MultinomialRaysampler succeeds GridRaysampler bringing masking and subsampling. Correspondingly, NDCMultinomialRaysampler succeeds NDCGridRaysampler.

Reviewed By: nikhilaravi, shapovalov

Differential Revision: D33256897

fbshipit-source-id: cd80ec6f35b110d1d20a75c62f4e889ba8fa5d45

Summary: Update all FB license strings to the new format.

Reviewed By: patricklabatut

Differential Revision: D33403538

fbshipit-source-id: 97a4596c5c888f3c54f44456dc07e718a387a02c

Summary: Removed harmonic embedding function from projects/nerf and changed import to be from core pytorch3d.

Reviewed By: patricklabatut

Differential Revision: D33142358

fbshipit-source-id: 3004247d50392dbd04ea72e9cd4bace0dc03606b

Summary: PyTorch 1.6.0 came out on 28 Jul 2020. Stop builds for 1.5.0 and 1.5.1. Also update the news section of the README for recent releases.

Reviewed By: nikhilaravi

Differential Revision: D31442830

fbshipit-source-id: 20bdd8a07090776d0461240e71c6536d874615f6

Summary: In D30349234 (https://github.com/facebookresearch/pytorch3d/commit/1b8d86a104eab24ac25863c423d084d611f64bae) we introduced persistent=False to some register_buffer calls, which depend on PyTorch 1.6. We go back to the old behaviour for PyTorch 1.5.

Reviewed By: nikhilaravi

Differential Revision: D30731327

fbshipit-source-id: ab02ef98ee87440ef02479b72f4872b562ab85b5

Summary:
As suggested in #802. By not persisting the _xy_grid buffer, we can allow (in some cases) a model with one image_size to be loaded from a saved model which was trained at a different resolution.

Also avoid persisting _frequencies in HarmonicEmbedding for similar reasons.

BC-break: This will cause load_state_dict, in strict mode, to complain if you try to load an old model with the new code.

Reviewed By: patricklabatut

Differential Revision: D30349234

fbshipit-source-id: d6061d1e51c9f79a78d61a9f732c9a5dfadbbb47

Summary:
Use PyTorch3D's new faster sample_pdf function instead of local Python implementation.

Also clarify deps for the Python implementation.

Reviewed By: gkioxari

Differential Revision: D30512109

fbshipit-source-id: 84cfdc00313fada37a6b29837de96f6a4646434f

Summary: Change doc references to master branch to its new name main.

Reviewed By: nikhilaravi

Differential Revision: D30303018

fbshipit-source-id: cfdbb207dfe3366de7e0ca759ed56f4b8dd894d1

Summary: Remove `pyre-fixme` and `pyre-ignore` and fix the type errors.

Reviewed By: kandluis

Differential Revision: D29899546

fbshipit-source-id: dc8314f314bbc8acc002b8dbf21013cf3bafe65d

Summary: More type annotations: device, shaders, pluggable I/O, stats in NeRF project, cameras, textures, etc...

Reviewed By: nikhilaravi

Differential Revision: D29327396

fbshipit-source-id: cdf0ceaaa010e22423088752688c8dd81f1acc3c

Summary: Annotate the (return type of the) following dunder functions across the codebase: `__init__()`, `__len__()`, `__getitem__()`

Reviewed By: nikhilaravi

Differential Revision: D29001801

fbshipit-source-id: 928d9e1c417ffe01ab8c0445311287786e997c7c

Summary: License lint codebase

Reviewed By: theschnitz

Differential Revision: D29001799

fbshipit-source-id: 5c59869911785b0181b1663bbf430bc8b7fb2909

Summary: If we are not visualizing the training with visdom, then there are a couple of outputs of the coarse rendering step which are not small and are returned by the renderer but never used. We don't need to bother transferring them to the CPU.

Reviewed By: nikhilaravi

Differential Revision: D28939958

fbshipit-source-id: 7e0d6681d6524f7fb57b6b20164580006120de80

Summary:
All classes implicitly inherit from `object` since Python 3, so remove unnecessary explicit inheritance.

From the [official documentation](https://docs.python.org/3/library/functions.html#object):

> `object` is a base for all classes.

Reviewed By: nikhilaravi

Differential Revision: D28942595

fbshipit-source-id: 466c0d19d8a93a6263e7ad734c3e87160cfa6066

Summary:
Inside the implicit function, the color and density calculations are independent and time is saved by putting them on separate streams.

(In fact, colors is slower than densities, and the raymarcher does some calculation with the densities before the colors. So theoretically we could go further and not join the streams together until the colors are actually needed. The code would be more complicated. But the profile suggests that the raymarcher is quick and so this wouldn't be expected to make a big difference.)

In inference, this might increase memory usage, so it isn't an obvious win. That is why I have added a flag.

Reviewed By: nikhilaravi

Differential Revision: D28648549

fbshipit-source-id: c087de80d8ccfce1dad3a13e71df2f305a36952e

Summary: When harmonic embedding is used, we always cat its input onto its output before proceeding. Avoid an intermediate tensor by making the module do that for itself.

Reviewed By: davnov134

Differential Revision: D28185791

fbshipit-source-id: 98d92c94a918dd42e16cdadcaac71dabbc7de5c3

Summary:
Add custom layer to avoid repeating copied data for every ray position.

This should also save time in the backward pass because there are fewer multiplies with the weights.

Reviewed By: theschnitz

Differential Revision: D28382412

fbshipit-source-id: 1ba7356cd8520ebd598568ae503e47d31d3989eb

Summary: Add gitignore file to ignore data and checkpoints in the NeRF project.

Reviewed By: nikhilaravi

Differential Revision: D28382413

fbshipit-source-id: 747d69f4353a76a28acde8ba26a896cb2278f976

Summary: We don't use gradents of sample_pdf. Here we disable gradient calculation around calling it, instead of calling detach later. There's a theoretical speedup, but mainly this enables using sample_pdf implementations which don't support gradients.

Reviewed By: nikhilaravi

Differential Revision: D28057284

fbshipit-source-id: 8a9d5e73f18b34e1e4291028008e02973023638d