GitLab

Summary:
Added support for barycentric clipping in the C++/CUDA rasterization kernels which can be switched on/off via a rasterization setting.

Added tests and a benchmark to compare with the current implementation in PyTorch - for some cases of large image size/faces per pixel the cuda version is 10x faster.

Reviewed By: gkioxari

Differential Revision: D21705503

fbshipit-source-id: e835c0f927f1e5088ca89020aef5ff27ac3a8769

Summary:
C++/CUDA implementation of forward and backward passes for the sigmoid alpha blending function.

This is slightly faster than the vectorized implementation in Python, but more importantly uses less memory due to fewer tensors being created.

Reviewed By: gkioxari

Differential Revision: D19980671

fbshipit-source-id: 0779055d2c68b1f20fb0870e60046077ef4613ff

Summary: R2N2 returns R2N2's own renderings of ShapeNetCore models.

Reviewed By: nikhilaravi

Differential Revision: D22266988

fbshipit-source-id: 36e67bd06c6459773e6e5f654259166b579be36a

Summary: Adding a render function for R2N2.

Reviewed By: nikhilaravi

Differential Revision: D22230228

fbshipit-source-id: a9f588ddcba15bb5d8be1401f68d730e810b4251

Summary: Skeleton of R2N2 that for now only returns verts and faces extracted from ShapeNetCore v1.

Reviewed By: nikhilaravi

Differential Revision: D22203656

fbshipit-source-id: 00db6ac76bfdb76fdbc77a2087c34a3f0ff01e6a

Summary: Adding collate_batched_meshes for datasets.utils: takes in a list of dictionaries and merge them into one dictionary (while adding a merged mesh to the dictionary).

Reviewed By: nikhilaravi

Differential Revision: D22180404

fbshipit-source-id: f811f9a140f09638f355ad5739bffa6ee415819f

Summary: Additional functionality for renderer in ShapeNetCore: users can select which objects to render by specifying their model_ids, or users could choose to render several random objects in some categories, or users could specify indices of the objects in the loaded dataset. (currently doesn't support changing lighting, still investigating why lighting is causing instability in renderings)

Reviewed By: nikhilaravi

Differential Revision: D22179594

fbshipit-source-id: 74c49094ffa3ea2eb71de9451f9e5da5053d356d

Summary: Adding a renderer to ShapeNetCore (Note that the lights are currently turned off for the test; will investigate why lighting causes instability in rendering)

Reviewed By: nikhilaravi

Differential Revision: D22102673

fbshipit-source-id: a704756a1e93b61d5a879f0e5ee14ebcb0df49d7

Summary: When rendering meshes with Phong shading, interpolate_face_attributes was taking up a nontrivial fraction of the overall shading time. This diff replaces our Python implementation of this function with a CUDA implementation.

Reviewed By: nikhilaravi

Differential Revision: D21610763

fbshipit-source-id: 2bb362a28f698541812aeab539047264b125ebb8

Summary:
There is a bug in efficient PnP that incorrectly weights points. This fixes it.

The test does not pass for the previous version with the bug.

Reviewed By: shapovalov

Differential Revision: D22449357

fbshipit-source-id: f5a22081e91d25681a6a783cce2f5c6be429ca6a

Summary: Conversion to/from the 6D representation of rotation from the paper http://arxiv.org/abs/1812.07035 ; based on David’s implementation.

Reviewed By: davnov134

Differential Revision: D22234397

fbshipit-source-id: 9e25ee93da7e3a2f2068cbe362cb5edc88649ce0

Summary:
Fix:
* Scaling of point clouds for scalars
* save_ply compatible cat

Reviewed By: nikhilaravi

Differential Revision: D22298609

fbshipit-source-id: abe94a5b64baf325587202d20adfc36912cc1478

Summary: Added `sorted` argument to the `knn_points` function. This came up during the benchmarking against Faiss - sorting added extra memory usage. Match the memory usage of Faiss by making sorting optional.

Reviewed By: bottler, gkioxari

Differential Revision: D22329070

fbshipit-source-id: 0828ff9b48eefce99ce1f60089389f6885d03139

Summary:
Fix default setting of `max_faces_per_bin` and update mesh rasterization benchmark tests.
The previous setting of `max_faces_per_bin` was wrong and for larger mesh sizes and batch sizes it was causing a significant slow down due to an unecessarily large intermediate tensor being created.

Reviewed By: gkioxari

Differential Revision: D22301819

fbshipit-source-id: d5e817f5b917fb5633c9c6a8634b6c8ff65e3508

Summary:
Fixes the default setting of `max_points_per_bin` in `rasterize_points.py`. For large batches with large size pointclouds this was a causing the rasterizer to be very slow.

Expanded the pointcloud rendering benchmarks to include larger size pointclouds and fixed cuda synchronization issue in benchmark.

Reviewed By: gkioxari

Differential Revision: D22301185

fbshipit-source-id: 5077c1ba2c43d73efc1c659f0ec75959ceddf893

Summary: Adding support so that users can select which categories they would like to load with wordnet synset offsets or labels or a combination of both. ShapeNetCore now also supports loading v2.

Reviewed By: nikhilaravi

Differential Revision: D22039207

fbshipit-source-id: 1f0218acb790e5561e2ae373e99cebb9823eea1a

Summary: Skeleton of ShapeNetCore class that loads ShapeNet v1 from a given directory to a Dataset object. Overrides _init_, _len_, and _getitem_ from torch.utils.data.Dataset. Currently getitem returns verts, faces and id_str, where id_str is a concatenation of synset_id and obj_id. Planning on adding support for loading ShapeNet v2, retrieving textures and returning wordnet synsets (not just ids) in next diffs.

Reviewed By: nikhilaravi

Differential Revision: D21986222

fbshipit-source-id: c2c515303f1898b6c495b52cb53c74d691585326

Summary:
point_mesh functions were missing CPU implementations.
The indices returned are not always matching, possibly due to numerical instability.

Reviewed By: gkioxari

Differential Revision: D21594264

fbshipit-source-id: 3016930e2a9a0f3cd8b3ac4c94a92c9411c0989d

Summary:
Pytorch seems to be becoming stricter about integer tensors of shape `(1,)` on GPU and not allowing them to be used as `int`s. For example the following no longer works on pytorch master,
    foo = torch.tensor([3, 5, 3], device="cuda:0")
    torch.arange(10) + foo[0]
because this is the sum of tensors on different devices.

Here fix tests which recently broke because of this.

Reviewed By: nikhilaravi

Differential Revision: D21929745

fbshipit-source-id: 25374f70468d1c895372766f1a9dd61df0833957

Summary: Adding a function in pytorch3d.structures.meshes to join multiple meshes into a Meshes object representing a single mesh. The function currently ignores all textures.

Reviewed By: nikhilaravi

Differential Revision: D21876908

fbshipit-source-id: 448602857e9d3d3f774d18bb4e93076f78329823

Summary: To avoid pytorch warnings and future behaviour changes, stop using torch.div and / with tensors of integers.

Reviewed By: gkioxari, mruberry

Differential Revision: D21857955

fbshipit-source-id: fb9f3000f3d953352cdc721d2a5f73d3a4bbf4b7

Summary: Adds support to hard_rgb_blend and hard blending shaders in shader.py (HardPhongShader, HardGouraudShader, and HardFlatShader) for changing the background color on which objects are rendered

Reviewed By: nikhilaravi

Differential Revision: D21746062

fbshipit-source-id: 08001200f4339d6a69c52405c6b8f4cac9f3f56e

Summary:
Update the transform method in the mesh rasterizer class to use the new `update_padded` method on the `Meshes` class to directly update the mesh vertices.

Also added a benchmark.

Reviewed By: gkioxari

Differential Revision: D21700352

fbshipit-source-id: c330e4040c681729eb2cc7bdfd92fb4a51a1a7d6

Summary:
Three changes to Meshes

1. `num_verts_per_mesh` and `num_faces_per_mesh` are assigned at construction time and are returned without the need for `compute_packed`
2. `update_padded` updates `verts_padded` and shallow copies faces list and faces_padded and existing attributes from construction.
3. `padded_to_packed_idx` does not need `compute_packed`

Reviewed By: nikhilaravi

Differential Revision: D21653674

fbshipit-source-id: dc6815a2e2a925fe4a834fe357919da2b2c14527

Summary: Add ability to decode ply files which use types like int32.

Reviewed By: nikhilaravi

Differential Revision: D21639208

fbshipit-source-id: 0ede7d4aa353a6e940446680a18e7ac0c48fafee

Summary: I was trying to speed up the lighting computations, but my ideas didn't work. Even if that didn't work, we can at least commit the benchmarking script I wrote for diffuse and specular shading.

Reviewed By: nikhilaravi

Differential Revision: D21580171

fbshipit-source-id: 8b60c0284e91ecbe258b6aae839bd5c2bbe788aa

Summary: lg-zhang found the problem with the quadratic part of ePnP implementation: n262385 . It was caused by a coefficient returned from the linear equation solver being equal to exactly 0.0, which caused `sign()` to return 0, something I had not anticipated. I also made sure we avoid division by zero by clamping all relevant denominators.

Reviewed By: nikhilaravi, lg-zhang

Differential Revision: D21531200

fbshipit-source-id: 9eb2fa9d4f4f8f5f411d4cf1cffcc44b365b7e51

Summary:
Fixes the case where the rotation angle is exactly 0/PI.
Added a test for `so3_log_map(identity_matrix)`.

Reviewed By: nikhilaravi

Differential Revision: D21477078

fbshipit-source-id: adff804da97f6f0d4f50aa1f6904a34832cb8bfe

Summary: Fix to enable a mesh/point rasterizer to be initialized without having to specify the camera.

Reviewed By: jcjohnson, gkioxari

Differential Revision: D21362359

fbshipit-source-id: 4f84ea18ad9f179c7b7c2289ebf9422a2f5e26de

Summary: Add alignment modes for cubify operation.

Reviewed By: nikhilaravi

Differential Revision: D21393199

fbshipit-source-id: 7022044e591229a6ed5efc361fd3215e65f43f86

Summary: This has been failing intermittently

Reviewed By: nikhilaravi

Differential Revision: D21403157

fbshipit-source-id: 51b74d6c813b52effe72d14b565e250fcabbb463

Summary:
Ran the linter.
TODO: need to update the linter as per D21353065.

Reviewed By: bottler

Differential Revision: D21362270

fbshipit-source-id: ad0e781de0a29f565ad25c43bc94a19b1828c020

Summary:
Use nn.functional.interpolate instead of a TorchVision transform to resize texture maps to a common value. This works on all devices. This fixes issue #175.

Also fix the condition so it only happens when needed.

Reviewed By: nikhilaravi

Differential Revision: D21324510

fbshipit-source-id: c50eb06514984995bd81f2c44079be6e0b4098e4

Summary: Fix self assignment of normals when estimating normals

Reviewed By: nikhilaravi

Differential Revision: D21315980

fbshipit-source-id: 2aa5864c3f066e39e07343f192cc6423ce1ae771

Summary: Bump the nvidia driver used in the conda tests. Add an environment variable (unused) to allow building without ninja. Print relative error on assertClose failure.

Reviewed By: nikhilaravi

Differential Revision: D21227373

fbshipit-source-id: 5dd8eb097151da27d3632daa755a1e7b9ac97845

Summary:
Cuda test failing on circle with the error `random_ expects 'from' to be less than 'to', but got from=0 >= to=0`

This is because the `high` value in `torch.randint` is 1 more than the highest value in the distribution from which a value is drawn. So if there is only 1 cuda device available then the low and high are 0.

Reviewed By: gkioxari

Differential Revision: D21236669

fbshipit-source-id: 46c312d431c474f1f2c50747b1d5e7afbd7df3a9

Summary:
Signed-off-by: Michele Sanna <sanna@arrival.com>

fixes the bin_size calculation with a formula for any image_size > 64. Matches the values chosen so far.

simple test:

```
import numpy as np
import matplotlib.pyplot as plt

image_size = np.arange(64, 2048)
bin_size = np.where(image_size <= 64, 8, (2 ** np.maximum(np.ceil(np.log2(image_size)) - 4, 4)).astype(int))

print(image_size)
print(bin_size)

for ims, bins in zip(image_size, bin_size):
    if ims <= 64:
        assert bins == 8
    elif ims <= 256:
        assert bins == 16
    elif ims <= 512:
        assert bins == 32
    elif ims <= 1024:
        assert bins == 64
    elif ims <= 2048:
        assert bins == 128

    assert (ims + bins - 1) // bins < 22

plt.plot(image_size, bin_size)
plt.grid()
plt.show()
```

![img](https://user-images.githubusercontent.com/54891577/75464693-795bcf00-597f-11ea-9061-26440211691c.png)
Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/90

Reviewed By: jcjohnson

Differential Revision: D21160372

Pulled By: nikhilaravi

fbshipit-source-id: 660cf5832f4ca5be243c435a6bed969596fc0188

Summary:
Updates to:
- enable cuda kernel launches on any GPU (not just the default)
- cuda and contiguous checks for all kernels
- checks to ensure all tensors are on the same device
- error reporting in the cuda kernels
- cuda tests now run on a random device not just the default

Reviewed By: jcjohnson, gkioxari

Differential Revision: D21215280

fbshipit-source-id: 1bedc9fe6c35e9e920bdc4d78ed12865b1005519

Summary:
Updated the load obj function to support creating of a per face texture map using the information in an .mtl file. Uses the approach from in SoftRasterizer.

Currently I have ported in the SoftRasterizer code but this is only to help with comparison and will  be deleted before landing. The ShapeNet Test data will also be deleted.

Here is the [Design doc](https://docs.google.com/document/d/1AUcLP4QwVSqlfLAUfbjM9ic5vYn9P54Ha8QbcVXW2eI/edit?usp=sharing).

## Added
- texture atlas creation functions in PyTorch based on the SoftRas cuda implementation
- tests to compare SoftRas vs PyTorch3D implementation to verify it matches (using real shapenet data with meshes consisting of multiple textures)
- benchmarks tests

## Remaining todo:
- add more tests for obj io to test the new functions and the two texturing options
- replace the shapenet data with the output from SoftRas saved as a file.

# MAIN FILES TO REVIEW

- `obj_io.py`
- `test_obj_io.py` [still some tests to be added but have comparisons with SoftRas for now]

The reference SoftRas implementations are in `softras_load_obj.py` and `load_textures.cu`.

Reviewed By: gkioxari

Differential Revision: D20754859

fbshipit-source-id: 42ace9dfb73f26e29d800c763f56d5b66c60c5e2

Summary:
We have multiple KNN CUDA implementations. From python, users can currently request a particular implementation via the `version` flag, but they have no way of knowing which implementations can be used for a given problem.

This diff exposes a function `pytorch3d._C.knn_check_version(version, D, K)` that returns whether a particular version can be used.

Reviewed By: nikhilaravi

Differential Revision: D21162573

fbshipit-source-id: 6061960bdcecba454fd920b00036f4e9ff3fdbc0