GitLab

Summary: New versions of cuda etc. I haven't committed recent changes to this for a while

Reviewed By: shapovalov

Differential Revision: D47396136

fbshipit-source-id: d6c27f5056fa8f4a74a628fa1d831159000acf55

Summary: This is needed from september 2023. As a side effect, implicitron docs should build better because typing.get_args exists etc.

Reviewed By: shapovalov

Differential Revision: D47363855

fbshipit-source-id: a954c5b81b1e5a4435fca146a11aea0d2ca96f45

Summary:
Blender uses OpenEXR to dump depth maps, so we have to support it.
OpenCV requires to explicitly accepth the vulnerabilities by setting the env var before exporting.
We can set it but I think it should be user’s responsibility.
OpenCV error reporting is adequate, so I don’t handle the error on our side.

Reviewed By: bottler

Differential Revision: D47403884

fbshipit-source-id: 2fcadd1df9d0efa0aea563bcfb2e3180b3c4d1d7

Summary:
For fg-masking depth, we assumed np.array but passed a Tensor; for defining the default depth_mask, vice versa.

Note that we change the intended behaviour for the latter, assuming that 0s are areas with empty depth. When loading depth masks, we replace NaNs with zeros, so it is sensible. It is not a BC change as that branch would crash if executed. Since there was no reports, I assume no one cared.

Reviewed By: bottler

Differential Revision: D47403588

fbshipit-source-id: 1094104176d7d767a5657b5bbc9f5a0cc9da0ede

Summary:
Convert ImplicitronRayBundle to a "classic" class instead of a dataclass. This change is introduced as a way to preserve the ImplicitronRayBundle interface while allowing two outcomes:
- init lengths arguments is now a Optional[torch.Tensor] instead of torch.Tensor
- lengths is now a property which returns a `torch.Tensor`. The lengths property will either recompute lengths from bins or return the stored _lengths. `_lenghts` is None if bins is set. It saves us a bit of memory.

Reviewed By: shapovalov

Differential Revision: D46686094

fbshipit-source-id: 3c75c0947216476ebff542b6f552d311024a679b

Summary:
## Context

Bins are used in mipnerf to allow to manipulate easily intervals. For example, by doing the following, `bins[..., :-1]` you will obtain all the left coordinates of your intervals, while doing `bins[..., 1:]` is equals to the right coordinates of your intervals.

We introduce here the support of bins like in MipNerf implementation.

## RayPointRefiner

Small changes have been made to modify RayPointRefiner.
- If bins is None

```
mids = torch.lerp(ray_bundle.lengths[..., 1:], ray_bundle.lengths[…, :-1], 0.5)
z_samples = sample_pdf(
		mids, # [..., npt]
		weights[..., 1:-1], # [..., npt - 1]
               ….
            )
```

- If bins is not None
In the MipNerf implementation the sampling is done on all the bins. It allows us to use the full weights tensor without slashing it.

```
z_samples = sample_pdf(
		ray_bundle.bins, # [..., npt + 1]
		weights, # [..., npt]
               ...
            )
```

## RayMarcher

Add a ray_deltas optional argument. If None, keep the same deltas computation from ray_lengths.

Reviewed By: shapovalov

Differential Revision: D46389092

fbshipit-source-id: d4f1963310065bd31c1c7fac1adfe11cbeaba606

Summary:
Add blurpool has defined in [MIP-NeRF](https://arxiv.org/abs/2103.13415).
It has been added has an option for RayPointRefiner.

Reviewed By: shapovalov

Differential Revision: D46356189

fbshipit-source-id: ad841bad86d2b591a68e1cb885d4f781cf26c111

Summary: Add a new implicit module Integral Position Encoding based on [MIP-NeRF](https://arxiv.org/abs/2103.13415).

Reviewed By: shapovalov

Differential Revision: D46352730

fbshipit-source-id: c6a56134c975d80052b3a11f5e92fd7d95cbff1e

Summary:
Introduce methods to approximate the radii of conical frustums along rays as described in [MipNerf](https://arxiv.org/abs/2103.13415):

- Two new attributes are added to ImplicitronRayBundle: bins and radii. Bins is of size n_pts_per_ray + 1. It allows us to manipulate easily and n_pts_per_ray intervals. For example we need the intervals coordinates in the radii computation for \(t_{\mu}, t_{\delta}\). Radii are used to store the radii of the conical frustums.

- Add 3 new methods to compute the radii:
   - approximate_conical_frustum_as_gaussians: It computes the mean along the ray direction, the variance of the
      conical frustum  with respect to t and variance of the conical frustum with respect to its radius. This
      implementation follows the stable computation defined in the paper.
   - compute_3d_diagonal_covariance_gaussian: Will leverage the two previously computed variances to find the
     diagonal covariance of the Gaussian.
   - conical_frustum_to_gaussian: Mix everything together to compute the means and the diagonal covariances along
     the ray of the Gaussians.

- In AbstractMaskRaySampler, introduces the attribute `cast_ray_bundle_as_cone`. If False it won't change the previous behaviour of the RaySampler. However if True, the samplers will sample `n_pts_per_ray +1` instead of `n_pts_per_ray`. This points are then used to set the bins attribute of ImplicitronRayBundle. The support of HeterogeneousRayBundle has not been added since the current code does not allow it. A safeguard has been added to avoid a silent bug in the future.

Reviewed By: shapovalov

Differential Revision: D45269190

fbshipit-source-id: bf22fad12d71d55392f054e3f680013aa0d59b78

Reviewed By: bottler

Differential Revision: D47223471

fbshipit-source-id: 8bdabf2a69dd7aec7202141122a9c69220ba7ef1

Summary: simple

Reviewed By: shapovalov

Differential Revision: D46438865

fbshipit-source-id: 0f41cb3ddd7e7aca4513267d33299531f7e8d373

Summary: We now use unittest.mock

Reviewed By: shapovalov

Differential Revision: D45868799

fbshipit-source-id: cd1042dc2c49c82c7b9e024f761c496049a31beb

Summary: Make test work in isolation, and when run internally make it not try the sqlalchemy files.

Reviewed By: shapovalov

Differential Revision: D46352513

fbshipit-source-id: 7417a25d7a5347d937631c9f56ae4e3242dd622e

Summary:
Hi,

Not sure this is the best fix. But while running this notebook, I only ever saw a blank canvas when trying to visualize the dolphin. It might be that I have a broken dependency, like plotly. I also don't know what the visualization is "supposed" to look like.

But incase other people have this issue, this one line change solved the whole problem for me. Now I have a happy, rotatable dolphin.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1549

Reviewed By: shapovalov

Differential Revision: D46350930

Pulled By: bottler

fbshipit-source-id: e19aa71eb05a93e2955262a2c90d1f0d09576228

Summary: Fix for https://github.com/facebookresearch/pytorch3d/issues/1441 where we were indexing with a tensor on the wrong device.

Reviewed By: shapovalov

Differential Revision: D46276449

fbshipit-source-id: 7750ed45ffecefa5d291fd1eadfe515310c2cf0d

Summary: Making it easier for the clients to use these datasets.

Reviewed By: bottler

Differential Revision: D46727179

fbshipit-source-id: cf619aee4c4c0222a74b30ea590cf37f08f014cc

Summary: In D42739669, I forgot to update the API of existing implementations of DatasetBase to take `subset_filter`. Looks like only one was missing.

Reviewed By: bottler

Differential Revision: D46724488

fbshipit-source-id: 13ab7a457f853278cf06955aad0cc2bab5fbcce6

Summary:
The parameter was renamed in MPL 3.5.0 in 2021, and the deprecated version is no longer supported.

https://matplotlib.org/stable/api/prev_api_changes/api_changes_3.5.0.html#the-first-parameter-of-axes-grid-and-axis-grid-has-been-renamed-to-visible

Reviewed By: bottler

Differential Revision: D46724242

fbshipit-source-id: 04e2ab6d63369d939ea4f0ce7d64693e0b95ee91

Summary:
Adds stratified sampling of sequences within categories applied after category / sequence filters but before the num sequence limit.
It respects the insertion order into the sequence_annots table, i.e. takes top N sequences within each category.

Reviewed By: bottler

Differential Revision: D46724002

fbshipit-source-id: 597cb2a795c3f3bc07f838fc51b4e95a4f981ad3

Summary: Single directional chamfer distance and option to use non-absolute cosine similarity

Reviewed By: bottler

Differential Revision: D46593980

fbshipit-source-id: b2e591706a0cdde1c2d361614cecebb84a581433

Differential Revision: D46685078

fbshipit-source-id: daf2e75f24b68d2eab74cddca8ab9446e96951e7

Summary: fixes https://github.com/facebookresearch/pytorch3d/issues/1554 , needed for Matplotlib 3.6+

Reviewed By: patricklabatut

Differential Revision: D46438822

fbshipit-source-id: f3c06ad5d8e881a635edd14f96d498dca73c169f

Summary: Fine implicit function was called before the coarse implicit function.

Reviewed By: shapovalov

Differential Revision: D46224224

fbshipit-source-id: 6b1cc00cc823d3ea7a5b42774c9ec3b73a69edb5

Differential Revision: D46119066

fbshipit-source-id: b766118b9dcc07075d328fba73f272419dc9fc38

Summary:
1. We may need to store arrays of unknown shape in the database. It implements and tests serialisation.

2. Previously, when an inexisting metadata file was passed to SqlIndexDataset, it would try to open it and create an empty file, then crash. We now open the file in a read-only mode, so the error message is more intuitive. Note that the implementation is SQLite specific.

Reviewed By: bottler

Differential Revision: D46047857

fbshipit-source-id: 3064ae4f8122b4fc24ad3d6ab696572ebe8d0c26

Summary: I don't know why RE tests sometimes fail here, but maybe it's a race condition. If that's right, this should fix it.

Reviewed By: shapovalov

Differential Revision: D46020054

fbshipit-source-id: 20b746b09ad9bd77c2601ac681047ccc6cc27ed9

Summary:
This is mostly a refactoring diff to reduce friction in extending the frame data.

Slight functional changes: dataset getitem now accepts (seq_name, frame_number_as_singleton_tensor) as a non-advertised feature. Otherwise this code crashes:
```
item = dataset[0]
dataset[item.sequence_name, item.frame_number]
```

Reviewed By: bottler

Differential Revision: D45780175

fbshipit-source-id: 75b8e8d3dabed954a804310abdbd8ab44a8dea29

Summary: We don't want to use print directly in stats.print() method. Instead this method will return the output string to the caller.

Reviewed By: shapovalov

Differential Revision: D45356240

fbshipit-source-id: 2cabe3cdfb9206bf09aa7b3cdd2263148a5ba145

Summary: version number

Reviewed By: shapovalov

Differential Revision: D45704549

fbshipit-source-id: d63867f305b07c30ed9ea104f1494d23710fdbb7

Summary: Drop support for PyTorch 1.9.0 and 1.9.1.

Reviewed By: shapovalov

Differential Revision: D45704329

fbshipit-source-id: c0fe3ecf6a1eb9bcd4163785c0cb4bf4f5060f50

Summary: In refactoring, we lost path manager here, which broke manifold storage. Fixing this.

Reviewed By: bottler

Differential Revision: D45574940

fbshipit-source-id: 579349eaa654215a09e057be57b56b46769c986a

Summary:
typing.NamedTuple was simplified in 3.10
These two fields were the same in 3.8,  so this should be a no-op

#buildmore

Reviewed By: bottler

Differential Revision: D45373526

fbshipit-source-id: 2b26156f5f65b7be335133e9e705730f7254260d

Summary:
Although we can load per-vertex normals in `load_obj`, saving per-vertex normals is not supported in `save_obj`.

This patch fixes this by allowing passing per-vertex normal data in `save_obj`:
``` python
def save_obj(
    f: PathOrStr,
    verts,
    faces,
    decimal_places: Optional[int] = None,
    path_manager: Optional[PathManager] = None,
    *,
    verts_normals: Optional[torch.Tensor] = None,
    faces_normals: Optional[torch.Tensor] = None,
    verts_uvs: Optional[torch.Tensor] = None,
    faces_uvs: Optional[torch.Tensor] = None,
    texture_map: Optional[torch.Tensor] = None,
) -> None:
    """
    Save a mesh to an .obj file.

    Args:
        f: File (str or path) to which the mesh should be written.
        verts: FloatTensor of shape (V, 3) giving vertex coordinates.
        faces: LongTensor of shape (F, 3) giving faces.
        decimal_places: Number of decimal places for saving.
        path_manager: Optional PathManager for interpreting f if
            it is a str.
        verts_normals: FloatTensor of shape (V, 3) giving the normal per vertex.
        faces_normals: LongTensor of shape (F, 3) giving the index into verts_normals
            for each vertex in the face.
        verts_uvs: FloatTensor of shape (V, 2) giving the uv coordinate per vertex.
        faces_uvs: LongTensor of shape (F, 3) giving the index into verts_uvs for
            each vertex in the face.
        texture_map: FloatTensor of shape (H, W, 3) representing the texture map
            for the mesh which will be saved as an image. The values are expected
            to be in the range [0, 1],
    """
```

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1511

Reviewed By: shapovalov

Differential Revision: D45086045

Pulled By: bottler

fbshipit-source-id: 666efb0d2c302df6cf9f2f6601d83a07856bf32f

Reviewed By: bottler

Differential Revision: D45600232

fbshipit-source-id: f41b95c6fca86d241666b54755a128cd33f6dd32

Summary:
If my understanding is right, prp_screen[1] should be 32 rather than 48.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/1501

Reviewed By: shapovalov

Differential Revision: D45044406

Pulled By: bottler

fbshipit-source-id: 7dd93312db4986f4701e642ba82d94333466b921

Summary: Fix for https://github.com/facebookresearch/pytorch3d/issues/1529

Reviewed By: shapovalov

Differential Revision: D45569211

fbshipit-source-id: 8c485f26cd409cafac53d4d982a03cde81a1d853

Summary: Import generic path; avoiding incorrect path patching.

Reviewed By: bottler

Differential Revision: D45573976

fbshipit-source-id: e6ff4d759deb936e3b636defa1e0851fb0127b46

Summary: do like xformers.

Reviewed By: shapovalov

Differential Revision: D44541873

fbshipit-source-id: 2c23160591cd9026fcd4972998d1bc90adba1356

Summary:
I forgot to include these tests to D45086611 when transferring code from pixar_replay repo.

They test the new ORM types used in SQL dataset and are SQL Alchemy 2.0 specific.

An important test for extending types is a proof of concept for generality of SQL Dataset. The idea is to extend FrameAnnotation and FrameData in parallel.

Reviewed By: bottler

Differential Revision: D45529284

fbshipit-source-id: 2a634e518f580c312602107c85fc320db43abcf5

Summary:
Added a suit of functions and code additions to experimental_gltf_io.py file to enable saving Meshes in TexturesVertex format into .glb file.
Also added a test to tets_io_gltf.py to check the functionality with the test described in Test Plane.

Reviewed By: bottler

Differential Revision: D44969144

fbshipit-source-id: 9ce815a1584b510442fa36cc4dbc8d41cc3786d5