GitLab

Summary: Add facilities for dataloading non-sequential scenes.

Reviewed By: shapovalov

Differential Revision: D37291277

fbshipit-source-id: 0a33e3727b44c4f0cba3a2abe9b12f40d2a20447

Summary: Refactors autodecoders. Tests pass.

Reviewed By: bottler

Differential Revision: D37592429

fbshipit-source-id: 8f5c9eac254e1fdf0704d5ec5f69eb42f6225113

Summary:
Make ViewMetrics easy to replace by putting them into an OmegaConf dataclass.

Also, re-word a few variable names and fix minor TODOs.

Reviewed By: bottler

Differential Revision: D37327157

fbshipit-source-id: 78d8e39bbb3548b952f10abbe05688409fb987cc

Summary: After landing https://github.com/pytorch/pytorch/pull/69607, that made it an error to use indexing with `cpu_tensor[cuda_indices]`. There was one outstanding test in fbcode that incorrectly used indexing in that way, which is fixed here

Reviewed By: bottler, osalpekar

Differential Revision: D37128838

fbshipit-source-id: 611b6f717b5b5d89fa61fd9ebeb513ad7e65a656

Summary:
David had his code crashed when using frame_annot["meta"] dictionary. Turns out we had a typo.
The tests were passing by chance since all the keys were single-character strings.

Reviewed By: bottler

Differential Revision: D37503987

fbshipit-source-id: c12b0df21116cfbbc4675a0182b9b9e6d62bad2e

Summary:
X-link: https://github.com/fairinternal/pytorch3d/pull/36

This adds two shaders for rendering depth maps for meshes. This is useful for structure from motion applications that learn depths based off of camera pair disparities.

There's two shaders, one hard which just returns the distances and then a second that does a cumsum on the probabilities of the points with a weighted sum. Areas that don't have any z faces are set to the zfar distance.

Output from this renderer is `[N, H, W]` since it's just depth no need for channels.

I haven't tested this in an ML model yet just in a notebook.

hard:
![hardzshader](https://user-images.githubusercontent.com/909104/170190363-ef662c97-0bd2-488c-8675-0557a3c7dd06.png)

soft:
![softzshader](https://user-images.githubusercontent.com/909104/170190365-65b08cd7-0c49-4119-803e-d33c1d8c676e.png)

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1208

Reviewed By: bottler

Differential Revision: D36682194

Pulled By: d4l3k

fbshipit-source-id: 5d4e10c6fb0fff5427be4ddd3bd76305a7ccc1e2

Summary: Pull Request resolved: https://github.com/fairinternal/pytorch3d/pull/38

Reviewed By: ignacio-rocco

Differential Revision: D37415027

Pulled By: bottler

fbshipit-source-id: 9b17049e4762506cd5c152fd6e244d5f0d97855b

Summary: OmegaConf 2.2.2 doesn't like heterogenous tuples or Sequence or Set members. Workaround this.

Reviewed By: shapovalov

Differential Revision: D37278736

fbshipit-source-id: 123e6657947f5b27514910e4074c92086a457a2a

Summary: small followup to D37172537 (https://github.com/facebookresearch/pytorch3d/commit/cba26506b6fe8a98695f50673cb20d9597d87551) and D37209012 (https://github.com/facebookresearch/pytorch3d/commit/81d63c63823e146e74d7be367d19314ab16d6815): changing default #harmonics and improving a test

Reviewed By: shapovalov

Differential Revision: D37412357

fbshipit-source-id: 1af1005a129425fd24fa6dd213d69c71632099a0

Summary: The blender synthetic dataset contains object masks in the alpha channel. Provide these in the corresponding dataset.

Reviewed By: shapovalov

Differential Revision: D37344380

fbshipit-source-id: 3ddacad9d667c0fa0ae5a61fb1d2ffc806c9abf3

Summary: Images were coming out in the wrong format.

Reviewed By: shapovalov

Differential Revision: D37291278

fbshipit-source-id: c10871c37dd186982e7abf2071ac66ed583df2e6

Summary: Just register ImplicitronDataSource. We don't use it as pluggable yet here.

Reviewed By: shapovalov

Differential Revision: D37315698

fbshipit-source-id: ac41153383f9ab6b14ac69a3dfdc44aca0d94995

Summary: Document the inputs of idr functions and distinguish n_harmonic_functions to be 0 (simple embedding) versus -1 (no embedding).

Reviewed By: davnov134

Differential Revision: D37209012

fbshipit-source-id: 6e5c3eae54c4e5e8c3f76cad1caf162c6c222d52

Summary: Use generator.permutation instead of choice so that different options for n_known_frames_for_test are nested.

Reviewed By: davnov134

Differential Revision: D37210906

fbshipit-source-id: fd0d34ce62260417c3f63354a3f750aae9998b0d

Summary: Allow specifying a color for non-opaque pixels in LSTMRenderer.

Reviewed By: davnov134

Differential Revision: D37172537

fbshipit-source-id: 6039726678bb7947f7d8cd04035b5023b2d5398c

Summary: Copy code from NeRF for loading LLFF data and blender synthetic data, and create dataset objects for them

Reviewed By: shapovalov

Differential Revision: D35581039

fbshipit-source-id: af7a6f3e9a42499700693381b5b147c991f57e5d

Differential Revision: D37172764

fbshipit-source-id: a2ec367e56de2781a17f5e708eb5832ec9d7e6b4

Summary:
pyfmt now specifies a target Python version of 3.8 when formatting
with black. With this new config, black adds trailing commas to all
multiline function calls. This applies the new formatting as part
of rolling out the linttool-integration for pyfmt.

paintitblack

Reviewed By: zertosh, lisroach

Differential Revision: D37084377

fbshipit-source-id: 781a1b883a381a172e54d6e447137657977876b4

Summary: Add test that the yaml files deserialize.

Reviewed By: davnov134

Differential Revision: D36830673

fbshipit-source-id: b785d8db97b676686036760bfa2dd3fa638bda57

Summary: Preparing for pluggables in experiment.py

Reviewed By: davnov134

Differential Revision: D36830674

fbshipit-source-id: eab499d1bc19c690798fbf7da547544df7e88fa5

Summary: Make dataset type and args configurable on JsonIndexDatasetMapProvider.

Reviewed By: davnov134

Differential Revision: D36666705

fbshipit-source-id: 4d0a3781d9a956504f51f1c7134c04edf1eb2846

Summary: Allow access to manifold internally by default.

Reviewed By: davnov134

Differential Revision: D36760481

fbshipit-source-id: 2a16bd40e81ef526085ac1b3f4606b63c1841428

Summary: Changes to JsonIndexDataset to make it fit with OmegaConf.structured. Also match some default values to what the provider defaults to.

Reviewed By: davnov134

Differential Revision: D36666704

fbshipit-source-id: 65b059a1dbaa240ce85c3e8762b7c3db3b5a6e75

Summary: Allow a class to modify its subparts in get_default_args by defining the special function provide_config_hook.

Reviewed By: davnov134

Differential Revision: D36671081

fbshipit-source-id: 3e5b73880cb846c494a209c4479835f6352f45cf

Summary: New paths.

Reviewed By: patricklabatut

Differential Revision: D36734929

fbshipit-source-id: c0ce7ee9145ddca07ef3758d31cc3c261b088e7d

Summary: Our tests fail (https://fburl.com/jmoqo9bz) because test_splatter_blend uses torch.tile, which is not supported in earlier torch versions. Replace it with tensor.extend.

Reviewed By: bottler

Differential Revision: D36796098

fbshipit-source-id: 38d5b40667f98f3163b33f44e53e96b858cfeba2

Summary: As subj.

Reviewed By: bottler

Differential Revision: D36705775

fbshipit-source-id: 7370710e863025dc07a140b41f77a7c752e3159f

Summary: Fix divide by zero for empty pointcloud in chamfer. Also for empty batches. In process, needed to regularize num_points_per_cloud for empty batches.

Reviewed By: kjchalup

Differential Revision: D36311330

fbshipit-source-id: 3378ab738bee77ecc286f2110a5c8dc445960340

Summary: A few minor additions I didn't fit into the SplatterBlender diffs, as requested by reviewers.

Reviewed By: jcjohnson

Differential Revision: D36682437

fbshipit-source-id: 57af995e766dfd2674b3984a3ba00aef7ca7db80

Summary: Add simple interactive testrunner for experiment.py

Reviewed By: shapovalov

Differential Revision: D35316221

fbshipit-source-id: d424bcba632eef89eefb56e18e536edb58ec6f85

Summary: The ImplicitronDataset class corresponds to JsonIndexDatasetMapProvider

Reviewed By: shapovalov

Differential Revision: D36661396

fbshipit-source-id: 80ca2ff81ef9ecc2e3d1f4e1cd14b6f66a7ec34d

Summary: Move testing targets from pytorch3d/tests/TARGETS to pytorch3d/TARGETS.

Reviewed By: shapovalov

Differential Revision: D36186940

fbshipit-source-id: a4c52c4d99351f885e2b0bf870532d530324039b

Summary: test_viewpool was inactive so missed being fixed in D36547815 (https://github.com/facebookresearch/pytorch3d/commit/2d1c6d5d9382651bbf825f5f1677d576305d1f92)

Reviewed By: kjchalup

Differential Revision: D36625587

fbshipit-source-id: e7224eadfa5581fe61f10f67d2221071783de04a

Summary:
Benchmarking. We only use num_faces=2 for splatter, because as far as I can see one would never need to use more. Pose optimization and mesh optimization experiments (see next two diffs) showed that Splatter with 2 faces beats Softmax with 50 and 100 faces in terms of accuracy.

Results: We're slower at 64px^2. At 128px and 256px, we're slower than Softmax+50faces, but faster than Softmax+100faces. We're also slower at 10 faces/pix, but expectation as well as results show that more then 2 faces shouldn't be necessary. See also more results in .https://fburl.com/gdoc/ttv7u7hp

Reviewed By: jcjohnson

Differential Revision: D36210575

fbshipit-source-id: c8de28c8a59ce5fe21a47263bd43d2757b15d123

Summary: Splatting shader. See code comments for details. Same API as SoftPhongShader.

Reviewed By: jcjohnson

Differential Revision: D36354301

fbshipit-source-id: 71ee37f7ff6bb9ce028ba42a65741424a427a92d

Summary: If you miss grid_sample in recent pytorch, it gives a warning, so stop doing this.

Reviewed By: kjchalup

Differential Revision: D36410619

fbshipit-source-id: 41dd4455298645c926f4d96c2084093b3f64ee2c

Summary: PLY with mixture of triangle and quadrilateral faces was failing.

Reviewed By: gkioxari

Differential Revision: D36592981

fbshipit-source-id: 5373edb2f38389ac646a75fd2e1fa7300eb8d054

Summary: Use small image size for test_all_gm_configs

Reviewed By: shapovalov

Differential Revision: D36511528

fbshipit-source-id: 2c65f518a4f23626850343a62d103f85abfabd88

Summary: If no view pooling, don't disable image_feature_extractor. Make image_feature_extractor default to absent.

Reviewed By: davnov134

Differential Revision: D36547815

fbshipit-source-id: e51718e1bcbf65b8b365a6e894d4324f136635e9

Summary: Just a rename

Reviewed By: shapovalov

Differential Revision: D36516885

fbshipit-source-id: 2126e3aee26d89a95afdb31e06942d61cbe88d5a