GitLab

Summary: More helpful errors when the output channels aren't 1 for density and 3 for color

Reviewed By: shapovalov

Differential Revision: D40341088

fbshipit-source-id: 6074bf7fefe11c8e60fee4db2760b776419bcfee

Summary: Couldn't build p3d on devfair because C++17 is unsupported. Two structured bindings sneaked in.

Reviewed By: bottler

Differential Revision: D40280967

fbshipit-source-id: 9627f3f9f76247a6cefbeac067fdead67c6f4e14

Summary:
TensoRF at step 2000 does volume croping and resizing.
At those steps it calculates part of the voxel grid which has density big enough to have objects and resizes the grid to fit that object.
Change is done on 3 levels:
- implicit function subscribes to epochs and at specific epochs finds the bounding box of the object and calls resizing of the color and density voxel grids to fit it
- VoxelGrid module calls cropping of the underlaying voxel grid and resizing to fit previous size it also adjusts its extends and translation to match wanted size
- Each voxel grid has its own way of cropping the underlaying data

Reviewed By: kjchalup

Differential Revision: D39854548

fbshipit-source-id: 5435b6e599aef1eaab980f5421d3369ee4829c50

Summary:
avoid creating a numpy array of random things just to split it: this can now generate a warning e.g. if the list contains lists of varying lengths. There might also be a performance win here, and we could do more of the same if we care about that.

(The vanilla way to avoid the new warning is to replace `np.split(a,` with `np.split(np.array(a, dtype=object), ` btw.)

Reviewed By: shapovalov

Differential Revision: D40209308

fbshipit-source-id: daae33a23ceb444e8e7241f72ce1525593e2f239

Summary: Forward method is sped up using the scaffold, a low resolution voxel grid which is used to filter out the points in empty space. These points will be predicted as having 0 density and (0, 0, 0) color. The points which were not evaluated as empty space will be passed through the steps outlined above.

Reviewed By: kjchalup

Differential Revision: D39579671

fbshipit-source-id: 8eab8bb43ef77c2a73557efdb725e99a6c60d415

Summary: Avoid certain hardcoded paths in co3dv2 data

Reviewed By: davnov134

Differential Revision: D40209309

fbshipit-source-id: 0e83a15baa47d5bd07d2d23c6048cb4522c1ccba

Summary: Fix Circleci error: https://app.circleci.com/pipelines/github/facebookresearch/pytorch3d/1066/workflows/94df137b-4882-4959-8fe4-738af447db23/jobs/56560.

Reviewed By: kjchalup

Differential Revision: D40185409

fbshipit-source-id: 7121b0cae66bd60f718df2a5d9ef5d2ac3bc658c

Summary: Bugfix

Reviewed By: bottler

Differential Revision: D40146840

fbshipit-source-id: a8415c361ed3cd939999b87311aff4d3bb1bcfe1

Summary: Avoids use of `torch.cat` operation when rendering a volume by instead issuing multiple calls to `torch.nn.functional.grid_sample`. Density and color tensors can be large.

Reviewed By: bottler

Differential Revision: D40072399

fbshipit-source-id: eb4cd34f6171d54972bbf2877065f973db497de0

Summary:
Torch C++ extension for Marching Cubes

- Add torch C++ extension for marching cubes. Observe a speed up of ~255x-324x speed up (over varying batch sizes and spatial resolutions)

- Add C++ impl in existing unit-tests.

(Note: this ignores all push blocking failures!)

Reviewed By: kjchalup

Differential Revision: D39590638

fbshipit-source-id: e44d2852a24c2c398e5ea9db20f0dfaa1817e457

Summary: Overhaul of marching_cubes_naive for better performance and to avoid relying on unstable hashing. In particular, instead of hashing vertex positions, we index each interpolated vertex with its corresponding edge in the 3d grid.

Reviewed By: kjchalup

Differential Revision: D39419642

fbshipit-source-id: b5fede3525c545d1d374198928dfb216262f0ec0

Summary:
Threaded the for loop:
```
for (int yi = 0; yi < H; ++yi) {...}
```
in function `RasterizeMeshesNaiveCpu()`.
Chunk size is approx equal.

Reviewed By: bottler

Differential Revision: D40063604

fbshipit-source-id: 09150269405538119b0f1b029892179501421e68

Summary: Loads the whole dataset and moves it to the device and sends it to for sampling to enable full dataset heterogeneous raysampling.

Reviewed By: bottler

Differential Revision: D39263009

fbshipit-source-id: c527537dfc5f50116849656c9e171e868f6845b1

Summary:
Changed ray_sampler and metrics to be able to use mixed frame raysampling.

Ray_sampler now has a new member which it passes to the pytorch3d raysampler.
If the raybundle is heterogeneous metrics now samples images by padding xys first. This reduces memory consumption.

Reviewed By: bottler, kjchalup

Differential Revision: D39542221

fbshipit-source-id: a6fec23838d3049ae5c2fd2e1f641c46c7c927e3

Summary: new implicitronRayBundle with added cameraIDs and camera counts. Added to enable a single raybundle inside Implicitron and easier extension in the future. Since RayBundle is named tuple and RayBundleHeterogeneous is dataclass and RayBundleHeterogeneous cannot inherit RayBundle. So if there was no ImplicitronRayBundle every function that uses RayBundle now would have to use Union[RayBundle, RaybundleHeterogeneous] which is confusing and unecessary complicated.

Reviewed By: bottler, kjchalup

Differential Revision: D39262999

fbshipit-source-id: ece160e32f6c88c3977e408e966789bf8307af59

Summary:
Added heterogeneous raysampling to pytorch3d raysampler, different cameras are sampled different number of times.

 It now returns RayBundle if heterogeneous raysampling is off and new RayBundleHeterogeneous (with added fields `camera_ids` and `camera_counts`).  Heterogeneous raysampling is on if `n_rays_total` is not None.

Reviewed By: bottler

Differential Revision: D39542222

fbshipit-source-id: d3d88d822ec7696e856007c088dc36a1cfa8c625

Summary:
This is quite a thin wrapper – not sure we need it. The motivation is that `Transform3d` is not as matrix-centric now, it can be converted to SE(3) logarithm equally easily.

It simplifies things like averaging cameras and getting axis-angle of camera rotation (previously, one would need to call `se3_log_map(cameras.get_world_to_camera_transform().get_matrix())`), now one fewer thing to call / discover.

Reviewed By: bottler

Differential Revision: D39928000

fbshipit-source-id: 85248d5b8af136618f1d08791af5297ea5179d19

Summary:
`get_rotation_to_best_fit_xy` is useful to expose externally, however there was a bug (which we probably did not care about for our use case): it could return a rotation matrix with det(R) == −1.
The diff fixes that, and also makes centroid optional (it can be computed from points).

Reviewed By: bottler

Differential Revision: D39926791

fbshipit-source-id: 5120c7892815b829f3ddcc23e93d4a5ec0ca0013

Differential Revision: D39894833

fbshipit-source-id: 95a32d9cb352c5fea345c6947194ad52971d4fe9

Summary: Any module can be subscribed to step updates from the training loop. Once the training loop publishes a step the voxel grid changes its dimensions. During the construction of VoxelGridModule and its parameters it does not know which is the resolution that will be loaded from checkpoint, so before the checkpoint loading a hook runs which changes the VoxelGridModule's parameters to match shapes of the loaded checkpoint.

Reviewed By: bottler

Differential Revision: D39026775

fbshipit-source-id: 0d359ea5c8d2eda11d773d79c7513c83585d5f17

Summary:
User reported that cloned cameras fail to save. The error with latest PyTorch is

```
pickle.PicklingError: Can't pickle ~T_destination: attribute lookup T_destination on torch.nn.modules.module failed
```

This fixes it.

Reviewed By: btgraham

Differential Revision: D39692258

fbshipit-source-id: 75bbf3b8dfa0023dc28bf7d4cc253ca96e46a64d

Summary: Adding a checkerboard mesh utility to Pytorch3d.

Reviewed By: bottler

Differential Revision: D39718916

fbshipit-source-id: d43cd30e566b5db068bae6eed0388057634428c8

Summary:
We need to make packing/unpacking in 2 places for mixed frame raysampling (metrics and raysampler) but those tensors that need to be unpacked/packed have more than two dimensions.
I could have reshaped and stored dimensions but this seems to just complicate code there with something which packed_to_padded should support.
I could have made a separate function for implicitron but it would confusing to have two different padded_to_packed functions inside pytorch3d codebase one of which does packing for (b, max) and (b, max, f) and the other for (b, max, …)

Reviewed By: bottler

Differential Revision: D39729026

fbshipit-source-id: 2bdebf290dcc6c316b7fe1aeee49bbb5255e508c

Summary: The implicit function and its members and internal working

Reviewed By: kjchalup

Differential Revision: D38829764

fbshipit-source-id: 28394fe7819e311ed52c9defc9a1b29f37fbc495

Summary: Call expand_args_field when instantiating an object.

Reviewed By: shapovalov

Differential Revision: D39541931

fbshipit-source-id: de8e1038927ff0112463394412d5d8c26c4a1e17

Summary: Allow using the new `foreach` option on optimizers.

Reviewed By: shapovalov

Differential Revision: D39694843

fbshipit-source-id: 97109c245b669bc6edff0f246893f95b7ae71f90

Summary: Add the ability to process arbitrary point shapes `[n_grids, ..., 3]` instead of  only `[n_grids, n_points, 3]`.

Reviewed By: bottler

Differential Revision: D39574373

fbshipit-source-id: 0a9ecafe9ea58cd8f909644de43a1185ecf934f4

Summary:
Added export of UV textures to IO.save_mesh in Pytorch3d
MeshObjFormat now passes verts_uv, faces_uv, and texture_map as input to save_obj

TODO: check if TexturesUV.verts_uv_list or TexturesUV.verts_uv_padded() should be passed to save_obj

IO.save_mesh(obj_file, meshes, decimal_places=2) should be IO().save_mesh(obj_file, meshes, decimal_places=2)

Reviewed By: bottler

Differential Revision: D39617441

fbshipit-source-id: 4628b7f26f70e38c65f235852b990c8edb0ded23

Summary:
A significant speedup (e.g. >2% of a forward pass).

Move NDCMultinomialRaysampler parts of AbstractMaskRaySampler to members instead of living in a dict. The dict was hiding them from the nn.Module system so their _xy_grid members were remaining on the CPU. Therefore they were being copied to the GPU in every forward pass.

(We couldn't easily use a ModuleDict here because the enum keys are not strs.)

Reviewed By: shapovalov

Differential Revision: D39668589

fbshipit-source-id: 719b88e4a08fd7263a284e0ab38189e666bd7e3a

Summary:
- indicate location of OmegaConf.structured failures
- split the data gathering from enable_get_default_args to ease experimenting with it.
- comment fixes.
- nicer error when a_class_type has weird type.

Reviewed By: kjchalup

Differential Revision: D39434447

fbshipit-source-id: b80c7941547ca450e848038ef5be95b7ebbe8f3e

Summary: Fixes #1326

Reviewed By: kjchalup

Differential Revision: D39259697

fbshipit-source-id: 51392f4cc4a956165a62901cb115fcefe0e17277

Summary: Various fixes to get visualize_reconstruction running, and an interactive test for it.

Reviewed By: kjchalup

Differential Revision: D39286691

fbshipit-source-id: 88735034cc01736b24735bcb024577e6ab7ed336

Summary: allow TESTIT to complete properly

Reviewed By: kjchalup

Differential Revision: D39280546

fbshipit-source-id: 38fe69988a736e32dbe78d1d05e6d8421353854a

Summary: Workaround for oddity with new hydra.

Reviewed By: davnov134

Differential Revision: D39280639

fbshipit-source-id: 76e91947f633589945446db93cf2dbc259642f8a

Summary: Followup to D39259775 (https://github.com/facebookresearch/pytorch3d/commit/438c194ec61b34b377727c97d5922897546c581d)

Reviewed By: davnov134

Differential Revision: D39271753

fbshipit-source-id: 5cf11c1210369e1762ef0d5d0c7b60336711b261

Summary: Samples batches without replacement if the number of samples is not specified. This makes sure that we always iterate over the whole dataset in each epoch.

Reviewed By: bottler

Differential Revision: D39270786

fbshipit-source-id: 0c983d1f5e0af711463abfb23939bc0d2b5172a0

Summary: Fixes bugs in render_flyaround

Reviewed By: bottler

Differential Revision: D39271932

fbshipit-source-id: 07e6c9ee07ba91feb437b725af0a8942fd98db0b

Summary:
Move the flyaround rendering function into core implicitron.
The unblocks an example in the facebookresearch/co3d repo.

Reviewed By: bottler

Differential Revision: D39257801

fbshipit-source-id: 6841a88a43d4aa364dd86ba83ca2d4c3cf0435a4

Summary:
The self._stratified_sampling attribute is always overridden unless stratified_sampling is explicitly set to None. However, the desired default behavior is that the value of self._stratified_sampling is used unless the argument stratified_sampling is set to True/False. Changing the default to None achieves this

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1324

Reviewed By: bottler

Differential Revision: D39259775

Pulled By: davnov134

fbshipit-source-id: e01bb747ac80c812eb27bf22e67f5e14f29acadd

Summary: On each call of the stats.update the object calculates current average iteration time by getting time elapsed from the time_start and then dividing it by the current number of steps. It saves the result to AverageMeter object which when queried returns the average of things saved, so the time is averaged twice which biases it towards the start value (which is often larger).

Reviewed By: kjchalup

Differential Revision: D39206989

fbshipit-source-id: ccab5233d7aaca1ac4fd626fb329b83c7c0d6af9