GitLab

Summary:
Addresses the following issue:
https://github.com/facebookresearch/pytorch3d/issues/1345#issuecomment-1272881244

I.e., when installed from conda, `pytorch3d_implicitron_visualizer` crashes since it invokes `main()` while `main` requires a single positional arg `argv`.

Reviewed By: shapovalov

Differential Revision: D41533497

fbshipit-source-id: e53a923eb8b2f0f9c0e92e9c0866d9cb310c4799

Summary: Optional[some_configurable] won't autogenerate the enabled flag

Reviewed By: shapovalov

Differential Revision: D41522104

fbshipit-source-id: 555ff6b343faf6f18aad2f92fbb7c341f5e991c6

Summary: To be consistent with CUDA hashing, the diff replaces boost hasher with a simplified hasher for storing unique global edge_ids.

Reviewed By: kjchalup

Differential Revision: D41140382

fbshipit-source-id: 2ce598e5edcf6369fe13bd15d1f5e014b252027b

Summary:
Torch CUDA extension for Marching Cubes
- MC involving 3 steps:
  - 1st forward pass to collect vertices and occupied state for each voxel
  - Compute compactVoxelArray to skip non-empty voxels
  - 2nd pass to genereate interpolated vertex positions and faces by marching through the grid
- In contrast to existing MC:
   - Bind each interpolated vertex with a global edge_id to address floating-point precision
   - Added deduplication process to remove redundant vertices and faces

Benchmarks (ms):

| N / V(^3)      | python          | C++             |   CUDA   | Speedup |
| 2 / 20          |    12176873  |       24338     |     4363   | 2790x/5x|
| 1 / 100          |     -             |    3070511     |   27126   |    113x    |
| 2 / 100          |     -             |    5968934     |   53129   |    112x    |
| 1 / 256          |     -             |  61278092     | 430900   |    142x    |
| 2 / 256          |     -             |125687930     | 856941   |    146x   |

Reviewed By: kjchalup

Differential Revision: D39644248

fbshipit-source-id: d679c0c79d67b98b235d12296f383d760a00042a

Summary: Autogenerate docs for the renderer too. This will be helpful but make a slightly ugly TOC

Reviewed By: kjchalup

Differential Revision: D40977315

fbshipit-source-id: 10831de3ced68080cb5671c5dc31d4da8500f761

Summary:
Every time I try to run code, I get this warning:

```
  warnings.warn("Can't import pucuda.gl, not importing MeshRasterizerOpenGL.")
```

Of course, `pucuda` is a typo of `pycuda`.

This PR fixes the typo

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1379

Reviewed By: kjchalup

Differential Revision: D41295562

Pulled By: bottler

fbshipit-source-id: 2bfa2a2dbe20a5347861d36fbff5094994c1253d

Summary:
Enum fields cause the following to crash since they are loaded as strings:
```
config = OmegaConf.load(autodumped_cfg_file)
Experiment(**config)
```

It would be good to come up with the general solution but for now just fixing the visualisation script.

Reviewed By: bottler

Differential Revision: D41140426

fbshipit-source-id: 71c1c6b1fffe3b5ab1ca0114cfa3f0d81160278f

Summary:
Rasterize MC was not adapted to heterogeneous bundles.

There are some caveats though:
1) on CO3D, we get up to 18 points per image, which is too few for a reasonable visualisation (see below);
2) rasterising for a batch of 100 is slow.

I also moved the unpacking code close to the bundle to be able to reuse it.

{F789678778}

Reviewed By: bottler, davnov134

Differential Revision: D41008600

fbshipit-source-id: 9f10f1f9f9a174cf8c534b9b9859587d69832b71

Summary:
Allow a module's param_group member to specify overrides to the param groups of its members or their members.
Also logging for param group assignments.

This allows defining `params.basis_matrix` in the param_groups of a voxel_grid.

Reviewed By: shapovalov

Differential Revision: D41080667

fbshipit-source-id: 49f3b0e5b36e496f78701db0699cbb8a7e20c51e

Summary: Fix indexing of directions after filtering of points by scaffold.

Reviewed By: shapovalov

Differential Revision: D40853482

fbshipit-source-id: 9cfdb981e97cb82edcd27632c5848537ed2c6837

Summary:
Allows loading of multiple categories.
Multiple categories are provided in a comma-separated list of category names.

Reviewed By: bottler, shapovalov

Differential Revision: D40803297

fbshipit-source-id: 863938be3aa6ffefe9e563aede4a2e9e66aeeaa8

Summary: Quote formats, spelling

Reviewed By: shapovalov

Differential Revision: D40913734

fbshipit-source-id: d6dea65d5204b3c463c656a07ef9b447b7be6a0a

Summary: Fixes readthedocs. Sphinx build looks good.

Reviewed By: bottler

Differential Revision: D40893196

fbshipit-source-id: bf00384b921d4ef54e64745ed39172358c2f9bb3

Summary: Try to document implicitron. Most of this is autogenerated.

Reviewed By: shapovalov

Differential Revision: D40623742

fbshipit-source-id: 453508277903b7d987b1703656ba1ee09bc2c570

Summary: So that lpips isn't needed in readthedocs

Reviewed By: shapovalov

Differential Revision: D40859351

fbshipit-source-id: 58933bd3b71e78e658fbce56c3663b11ee2331f6

Summary: The bug lead to non-coinciding origins of the rays emitted from perspective cameras when unit_directions=True

Reviewed By: bottler

Differential Revision: D40865610

fbshipit-source-id: 398598e9e919b53e6bea179f0400e735bbb5b625

Summary: see title

Reviewed By: shapovalov

Differential Revision: D40803670

fbshipit-source-id: 211189167837af577d6502a698e2f3fb3aec3e30

Summary: Some things fail if a parameter is not wraped; in particular, it prevented other tensors moving to GPU.

Reviewed By: bottler

Differential Revision: D40819932

fbshipit-source-id: a23b38ceacd7f0dc131cb0355fef1178e3e2f7fd

Summary: installing from git: is failing

Reviewed By: shapovalov

Differential Revision: D40635668

fbshipit-source-id: 604ba5425e14caeabe4e178bf7f851f2163419bd

Differential Revision: D40695068

fbshipit-source-id: 9944a872d93a9dac348b252a0f8bf9c7e6f929f3

Summary: update version number

Reviewed By: shapovalov

Differential Revision: D40622583

fbshipit-source-id: 87fa55e1b02fc114f65ee8a5c3d998ba50226ab1

Summary: Yaml bool case fix

Reviewed By: shapovalov

Differential Revision: D40623031

fbshipit-source-id: 29b2fba171c2cbebfa03834e38b614d07275c997

Reviewed By: shapovalov

Differential Revision: D40622304

fbshipit-source-id: 277515a55c46d9b8300058b439526539a7fe00a0

Summary: Add option to flat pad the last delta. Might to help when training on rgb only.

Reviewed By: shapovalov

Differential Revision: D40587475

fbshipit-source-id: c763fa38948600ea532c730538dc4ff29d2c3e0a

Summary: Make Implicitron run without visdom installed.

Reviewed By: shapovalov

Differential Revision: D40587974

fbshipit-source-id: dc319596c7a4d10a4c54c556dabc89ad9d25c2fb

Summary:
According to the profiler trace D40326775, _check_valid_rotation_matrix is slow because of aten::all_close operation and _safe_det_3x3 bottlenecks. Disable the check by default unless environment variable PYTORCH3D_CHECK_ROTATION_MATRICES is set to 1.

Comparison after applying the change:
```
Profiling/Function    get_world_to_view (ms)   Transform_points(ms)    specular(ms)
before                12.751                    18.577                  21.384
after                 4.432 (34.7%)             9.248 (49.8%)           11.507 (53.8%)
```

Profiling trace:
https://pxl.cl/2h687
More details in https://docs.google.com/document/d/1kfhEQfpeQToikr5OH9ZssM39CskxWoJ2p8DO5-t6eWk/edit?usp=sharing

Reviewed By: kjchalup

Differential Revision: D40442503

fbshipit-source-id: 954b58de47de235c9d93af441643c22868b547d0

Summary: Keep the cause of hydra errors visible in some more cases.

Reviewed By: shapovalov

Differential Revision: D40516202

fbshipit-source-id: 8d214be5cc808a37738add77cc305fe099788546

Summary: param_groups only expected on MLPDecoder, not ElementwiseDecoder

Reviewed By: shapovalov

Differential Revision: D40508539

fbshipit-source-id: ea040ad6f7e26bd7d87e5de2eaadae2cf4b04faf

Summary:
Adds the ability to have different learning rates for different parts of the model. The trainable parts of the implicitron have a new member

       param_groups: dictionary where keys are names of individual parameters,
            or module’s members and values are the parameter group where the
            parameter/member will be sorted to. "self" key is used to denote the
            parameter group at the module level. Possible keys, including the "self" key
            do not have to be defined. By default all parameters are put into "default"
            parameter group and have the learning rate defined in the optimizer,
            it can be overriden at the:
                - module level with “self” key, all the parameters and child
                    module s parameters will be put to that parameter group
                - member level, which is the same as if the `param_groups` in that
                    member has key=“self” and value equal to that parameter group.
                    This is useful if members do not have `param_groups`, for
                    example torch.nn.Linear.
                - parameter level, parameter with the same name as the key
                    will be put to that parameter group.

And in the optimizer factory, parameters and their learning rates are recursively gathered.

Reviewed By: shapovalov

Differential Revision: D40145802

fbshipit-source-id: 631c02b8d79ee1c0eb4c31e6e42dbd3d2882078a

Summary:
Added initialization configuration for the last layer of the MLP decoding function. You can now set:
- last activation function (tensorf uses sigmoid)
- last bias init (tensorf uses 0, because of sigmoid ofc)
- option to use xavier initialization (we use relu so this should not be set)

Reviewed By: davnov134

Differential Revision: D40304981

fbshipit-source-id: ec398eb2235164ae85cb7c09b9660e843490ea04

Summary:
fix a typo

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1351

Reviewed By: shapovalov

Differential Revision: D40209834

Pulled By: bottler

fbshipit-source-id: 118133e0eab2df211e5c4f04371f2c695a9ceced

Summary: Restore the suppression of ffmpeg output.

Reviewed By: shapovalov

Differential Revision: D40296595

fbshipit-source-id: 41b2c14b6f6245f77e0ef6cc94fa7b41fbb83e33

Summary: Avoid unintended use of torch.range.

Reviewed By: kjchalup

Differential Revision: D40341396

fbshipit-source-id: 108295983afdec0ca9e43178fef9c65695150bc1

Summary:
Small config system fix. Allows get_default_args to work on an instance which has been created with a dict (instead of a DictConfig) as an args field. E.g.

```
gm = GenericModel(
        raysampler_AdaptiveRaySampler_args={"scene_extent": 4.0}
    )
    OmegaConf.structured(gm1)
```

Reviewed By: shapovalov

Differential Revision: D40341047

fbshipit-source-id: 587d0e8262e271df442a80858949a48e5d6db3df

Summary: Tensorf does relu or softmax after the density grid. This diff adds the ability to replicate that.

Reviewed By: bottler

Differential Revision: D40023228

fbshipit-source-id: 9f19868cd68460af98ab6e61c7f708158c26dc08

Summary: More helpful errors when the output channels aren't 1 for density and 3 for color

Reviewed By: shapovalov

Differential Revision: D40341088

fbshipit-source-id: 6074bf7fefe11c8e60fee4db2760b776419bcfee

Summary: Couldn't build p3d on devfair because C++17 is unsupported. Two structured bindings sneaked in.

Reviewed By: bottler

Differential Revision: D40280967

fbshipit-source-id: 9627f3f9f76247a6cefbeac067fdead67c6f4e14

Summary:
TensoRF at step 2000 does volume croping and resizing.
At those steps it calculates part of the voxel grid which has density big enough to have objects and resizes the grid to fit that object.
Change is done on 3 levels:
- implicit function subscribes to epochs and at specific epochs finds the bounding box of the object and calls resizing of the color and density voxel grids to fit it
- VoxelGrid module calls cropping of the underlaying voxel grid and resizing to fit previous size it also adjusts its extends and translation to match wanted size
- Each voxel grid has its own way of cropping the underlaying data

Reviewed By: kjchalup

Differential Revision: D39854548

fbshipit-source-id: 5435b6e599aef1eaab980f5421d3369ee4829c50

Summary:
avoid creating a numpy array of random things just to split it: this can now generate a warning e.g. if the list contains lists of varying lengths. There might also be a performance win here, and we could do more of the same if we care about that.

(The vanilla way to avoid the new warning is to replace `np.split(a,` with `np.split(np.array(a, dtype=object), ` btw.)

Reviewed By: shapovalov

Differential Revision: D40209308

fbshipit-source-id: daae33a23ceb444e8e7241f72ce1525593e2f239

Summary: Forward method is sped up using the scaffold, a low resolution voxel grid which is used to filter out the points in empty space. These points will be predicted as having 0 density and (0, 0, 0) color. The points which were not evaluated as empty space will be passed through the steps outlined above.

Reviewed By: kjchalup

Differential Revision: D39579671

fbshipit-source-id: 8eab8bb43ef77c2a73557efdb725e99a6c60d415