GitLab

Summary:
The look_at_view_transform did not give the correct results when the object location `at` was not (0,0,0).

The problem was on computing the cameras' location in world's coordinate `C`. It only took into account the camera position from spherical angles, but ignored the object location in the world's coordinate system. I simply modified the C tensor to take into account the object's location which is not necessarily in the origin.

I ran unit tests and all but 4 failed with the same error message: `RuntimeError: CUDA error: invalid device ordinal`. However the same happens before this patch, so I believe these errors are unrelated.

Pull Request resolved: https://github.com/facebookresearch/pytorch3d/pull/230

Reviewed By: gkioxari

Differential Revision: D23278126

Pulled By: nikhilaravi

fbshipit-source-id: c06e891bc46de8222325ee7b37aa43cde44648e8

Summary:
- Add support for loading textures from ShapeNet Obj files as a texture atlas.
- Support textured rendering of shapenet models

Reviewed By: gkioxari

Differential Revision: D23141143

fbshipit-source-id: 26eb81758d4cdbd6d820b072b58f5c6c08cb90bc

Summary:
Found a bug in extending textures with vertex uv coordinates. This was due to the padded -> list conversion of vertex uv coordinates i.e.                 The number of vertices in the mesh and in verts_uvs can differ
e.g. if a vertex is shared between 3 faces, it can
have up to 3 different uv coordinates. Therefore we cannot convert directly from padded to list using _num_verts_per_mesh

Reviewed By: bottler

Differential Revision: D23233595

fbshipit-source-id: 0c66d15baae697ead0bdc384f74c27d4c6539fc9

Summary:
faces_uvs_packed and verts_uvs_packed were only used in one place and the definition of the former was ambiguous. This meant that the wrong coordinates could be used for meshes other than the first in the batch. I have therefore removed both functions and build their common result inline. Added a test that a simple batch of two meshes is rendered consistently with the rendering of each alone. This test would have failed before.

I hope this fixes https://github.com/facebookresearch/pytorch3d/issues/283.

Some other small improvements to the textures code.

Reviewed By: nikhilaravi

Differential Revision: D23161936

fbshipit-source-id: f99b560a46f6b30262e07028b049812bc04350a7

Summary: A triangle is culled if any vertex in a triangle is behind the camera.  This fixes incorrect rendering of triangles that are partially behind the camera, where screen coordinate calculations are strange.  It doesn't work for triangles that are partially behind the camera but still intersect with the view frustum.

Reviewed By: nikhilaravi

Differential Revision: D22856181

fbshipit-source-id: a9cbaa1327d89601b83d0dfd3e4a04f934a4a213

Summary:
Refactor cameras
* CamerasBase was enhanced with `transform_points_screen` that transforms projected points from NDC to screen space
* OpenGLPerspective, OpenGLOrthographic -> FoVPerspective, FoVOrthographic
* SfMPerspective, SfMOrthographic -> Perspective, Orthographic
* PerspectiveCamera can optionally be constructred with screen space parameters
* Note on Cameras and coordinate systems was added

Reviewed By: nikhilaravi

Differential Revision: D23168525

fbshipit-source-id: dd138e2b2cc7e0e0d9f34c45b8251c01266a2063

Summary: Add `detach` for Meshes, Pointclouds, Textures

Reviewed By: nikhilaravi

Differential Revision: D23070418

fbshipit-source-id: 68671124ce114c4495d7ef3c944c9aac3d0db2d8

Summary:
Small fix to the softmax blending function.

To avoid overflow in the exponential for the softmax, the exponent is shifted by the maximum value. In the final calculation of the color there is a weighted sum between the pixel color and the background color - in order for the sum to be correct, the background color also needs to be handled in the same way witt the shifted exponent.

Reviewed By: gkioxari

Differential Revision: D23148301

fbshipit-source-id: 86066586ee7d3ce7bd4a2076b12ce191fbd151a7

Summary: Return R2N2's voxel coordinates.

Reviewed By: nikhilaravi

Differential Revision: D22462530

fbshipit-source-id: a995cfa0957b2561eb3b0f4591cb1db42170bc68

Summary: Return rotation, translation and intrinsic matrices necessary to reproduce R2N2's own renderings.

Reviewed By: nikhilaravi

Differential Revision: D22462520

fbshipit-source-id: 46a3859743ebc43c7a24f75827d2be3adf3f486b

Summary: The recently added part of a test was assuming that the random gpu was gpu 0.

Reviewed By: nikhilaravi

Differential Revision: D22948397

fbshipit-source-id: 88107e19fc3118e763f95be43a614941176a08f9

Summary: CPU implementation of the graph convolution op.

Reviewed By: nikhilaravi, gkioxari

Differential Revision: D21384361

fbshipit-source-id: bc96730e9727bb9aa1b0a232dcb82f0c0d12fe6b

Summary:
A fairly big refactor of the texturing API with some breaking changes to how textures are defined.

Main changes:
- There are now 3 types of texture classes: `TexturesUV`, `TexturesAtlas` and `TexturesVertex`. Each class:
   - has a `sample_textures` function which accepts the `fragments` from rasterization and returns `texels`. This means that the shaders will not need to know the type of the mesh texture which will resolve several issues people were reporting on GitHub.
  -  has a `join_batch` method for joining multiple textures of the same type into a batch

Reviewed By: gkioxari

Differential Revision: D21067427

fbshipit-source-id: 4b346500a60181e72fdd1b0dd89b5505c7a33926

Summary: Reduce the size of the data in this test, so that on circleci it doesn't run out of memory when pytorch (1.6) is used.

Reviewed By: gkioxari

Differential Revision: D22801490

fbshipit-source-id: 9591253c3d47430facd769a2c51a0b1722e0a305

Summary: Adding BlenderCamera (for rendering with R2N2 Blender transformations in the next diff).

Reviewed By: nikhilaravi

Differential Revision: D22462515

fbshipit-source-id: 4b40ee9bba8b6d56788dd3c723036ec704668153

Summary:
Sample/Get all views at the loading phase instead of returning phase;
Load only views from the split instead of all 24 views;
Test the numbers of views loaded are correct for each category.

Reviewed By: nikhilaravi

Differential Revision: D22631414

fbshipit-source-id: 1c5ce99fe2bdf6618c1aa0b69bb6899473376bc2

Summary: Fixing several unused imports and pyre/linter warnings.

Reviewed By: bottler

Differential Revision: D22592491

fbshipit-source-id: 463383b9b73a545949475044fb5c531712f8482c

Summary:
1. CircleCI tests fail because of different randomisation. I was able to reproduce it on devfair (with an older version of pytorch3d though), but with a new threshold, it works. Let’s push and see if it will work in CircleCI.
2. Fixing linter’s issue with `l` variable name.

Reviewed By: bottler

Differential Revision: D22573244

fbshipit-source-id: 32cebc8981883a3411ed971eb4a617469376964d

Summary:
Added support for barycentric clipping in the C++/CUDA rasterization kernels which can be switched on/off via a rasterization setting.

Added tests and a benchmark to compare with the current implementation in PyTorch - for some cases of large image size/faces per pixel the cuda version is 10x faster.

Reviewed By: gkioxari

Differential Revision: D21705503

fbshipit-source-id: e835c0f927f1e5088ca89020aef5ff27ac3a8769

Summary:
C++/CUDA implementation of forward and backward passes for the sigmoid alpha blending function.

This is slightly faster than the vectorized implementation in Python, but more importantly uses less memory due to fewer tensors being created.

Reviewed By: gkioxari

Differential Revision: D19980671

fbshipit-source-id: 0779055d2c68b1f20fb0870e60046077ef4613ff

Summary: R2N2 returns R2N2's own renderings of ShapeNetCore models.

Reviewed By: nikhilaravi

Differential Revision: D22266988

fbshipit-source-id: 36e67bd06c6459773e6e5f654259166b579be36a

Summary: Adding a render function for R2N2.

Reviewed By: nikhilaravi

Differential Revision: D22230228

fbshipit-source-id: a9f588ddcba15bb5d8be1401f68d730e810b4251

Summary: Skeleton of R2N2 that for now only returns verts and faces extracted from ShapeNetCore v1.

Reviewed By: nikhilaravi

Differential Revision: D22203656

fbshipit-source-id: 00db6ac76bfdb76fdbc77a2087c34a3f0ff01e6a

Summary: Adding collate_batched_meshes for datasets.utils: takes in a list of dictionaries and merge them into one dictionary (while adding a merged mesh to the dictionary).

Reviewed By: nikhilaravi

Differential Revision: D22180404

fbshipit-source-id: f811f9a140f09638f355ad5739bffa6ee415819f

Summary: Additional functionality for renderer in ShapeNetCore: users can select which objects to render by specifying their model_ids, or users could choose to render several random objects in some categories, or users could specify indices of the objects in the loaded dataset. (currently doesn't support changing lighting, still investigating why lighting is causing instability in renderings)

Reviewed By: nikhilaravi

Differential Revision: D22179594

fbshipit-source-id: 74c49094ffa3ea2eb71de9451f9e5da5053d356d

Summary: Adding a renderer to ShapeNetCore (Note that the lights are currently turned off for the test; will investigate why lighting causes instability in rendering)

Reviewed By: nikhilaravi

Differential Revision: D22102673

fbshipit-source-id: a704756a1e93b61d5a879f0e5ee14ebcb0df49d7

Summary: When rendering meshes with Phong shading, interpolate_face_attributes was taking up a nontrivial fraction of the overall shading time. This diff replaces our Python implementation of this function with a CUDA implementation.

Reviewed By: nikhilaravi

Differential Revision: D21610763

fbshipit-source-id: 2bb362a28f698541812aeab539047264b125ebb8

Summary:
There is a bug in efficient PnP that incorrectly weights points. This fixes it.

The test does not pass for the previous version with the bug.

Reviewed By: shapovalov

Differential Revision: D22449357

fbshipit-source-id: f5a22081e91d25681a6a783cce2f5c6be429ca6a

Summary: Conversion to/from the 6D representation of rotation from the paper http://arxiv.org/abs/1812.07035 ; based on David’s implementation.

Reviewed By: davnov134

Differential Revision: D22234397

fbshipit-source-id: 9e25ee93da7e3a2f2068cbe362cb5edc88649ce0

Summary:
Fix:
* Scaling of point clouds for scalars
* save_ply compatible cat

Reviewed By: nikhilaravi

Differential Revision: D22298609

fbshipit-source-id: abe94a5b64baf325587202d20adfc36912cc1478

Summary: Added `sorted` argument to the `knn_points` function. This came up during the benchmarking against Faiss - sorting added extra memory usage. Match the memory usage of Faiss by making sorting optional.

Reviewed By: bottler, gkioxari

Differential Revision: D22329070

fbshipit-source-id: 0828ff9b48eefce99ce1f60089389f6885d03139

Summary:
Fix default setting of `max_faces_per_bin` and update mesh rasterization benchmark tests.
The previous setting of `max_faces_per_bin` was wrong and for larger mesh sizes and batch sizes it was causing a significant slow down due to an unecessarily large intermediate tensor being created.

Reviewed By: gkioxari

Differential Revision: D22301819

fbshipit-source-id: d5e817f5b917fb5633c9c6a8634b6c8ff65e3508

Summary:
Fixes the default setting of `max_points_per_bin` in `rasterize_points.py`. For large batches with large size pointclouds this was a causing the rasterizer to be very slow.

Expanded the pointcloud rendering benchmarks to include larger size pointclouds and fixed cuda synchronization issue in benchmark.

Reviewed By: gkioxari

Differential Revision: D22301185

fbshipit-source-id: 5077c1ba2c43d73efc1c659f0ec75959ceddf893

Summary: Adding support so that users can select which categories they would like to load with wordnet synset offsets or labels or a combination of both. ShapeNetCore now also supports loading v2.

Reviewed By: nikhilaravi

Differential Revision: D22039207

fbshipit-source-id: 1f0218acb790e5561e2ae373e99cebb9823eea1a

Summary: Skeleton of ShapeNetCore class that loads ShapeNet v1 from a given directory to a Dataset object. Overrides _init_, _len_, and _getitem_ from torch.utils.data.Dataset. Currently getitem returns verts, faces and id_str, where id_str is a concatenation of synset_id and obj_id. Planning on adding support for loading ShapeNet v2, retrieving textures and returning wordnet synsets (not just ids) in next diffs.

Reviewed By: nikhilaravi

Differential Revision: D21986222

fbshipit-source-id: c2c515303f1898b6c495b52cb53c74d691585326

Summary:
point_mesh functions were missing CPU implementations.
The indices returned are not always matching, possibly due to numerical instability.

Reviewed By: gkioxari

Differential Revision: D21594264

fbshipit-source-id: 3016930e2a9a0f3cd8b3ac4c94a92c9411c0989d

Summary:
Pytorch seems to be becoming stricter about integer tensors of shape `(1,)` on GPU and not allowing them to be used as `int`s. For example the following no longer works on pytorch master,
    foo = torch.tensor([3, 5, 3], device="cuda:0")
    torch.arange(10) + foo[0]
because this is the sum of tensors on different devices.

Here fix tests which recently broke because of this.

Reviewed By: nikhilaravi

Differential Revision: D21929745

fbshipit-source-id: 25374f70468d1c895372766f1a9dd61df0833957

Summary: Adding a function in pytorch3d.structures.meshes to join multiple meshes into a Meshes object representing a single mesh. The function currently ignores all textures.

Reviewed By: nikhilaravi

Differential Revision: D21876908

fbshipit-source-id: 448602857e9d3d3f774d18bb4e93076f78329823

Summary: To avoid pytorch warnings and future behaviour changes, stop using torch.div and / with tensors of integers.

Reviewed By: gkioxari, mruberry

Differential Revision: D21857955

fbshipit-source-id: fb9f3000f3d953352cdc721d2a5f73d3a4bbf4b7

Summary: Adds support to hard_rgb_blend and hard blending shaders in shader.py (HardPhongShader, HardGouraudShader, and HardFlatShader) for changing the background color on which objects are rendered

Reviewed By: nikhilaravi

Differential Revision: D21746062

fbshipit-source-id: 08001200f4339d6a69c52405c6b8f4cac9f3f56e