Sphinx docs

Summary: Added sphinx docs and github workflow to build gh-pages

Reviewed By: HapeMask

Differential Revision: D63498319

fbshipit-source-id: 401aabb6dc1624a26c2c9231ab2a1e6b98458190

docs/source/api_reference/transform.rst

+drtk.transform
+==============
+
+.. automodule:: drtk.transform
+    :no-index:
+
+.. currentmodule:: drtk
+
+.. autosummary::
+    :nosignatures:
+
+    transform
+    transform_with_v_cam
+
+.. autofunction:: transform
+
+.. autofunction:: transform_with_v_cam

docs/source/conf.py

+#!/usr/bin/env python3
+
+# -- Path setup --------------------------------------------------------------
+
+import builtins
+import os
+import sys
+
+builtins.__sphinx_build__ = True
+
+current_dir = os.path.dirname(__file__)
+target_dir = os.path.abspath(os.path.join(current_dir, "../.."))
+sys.path.insert(0, target_dir)
+print(target_dir)
+from drtk import __version__
+
+# -- Project information -----------------------------------------------------
+
+project = "DRTK"
+copyright = "2024, Meta"
+author = "Meta"
+version = __version__
+
+# -- General configuration ---------------------------------------------------
+
+html_theme = "pydata_sphinx_theme"
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+    "sphinx_markdown_builder",
+    "sphinx.ext.autodoc",
+    "sphinx.ext.autosummary",
+    "sphinx.ext.viewcode",
+    "sphinx.ext.napoleon",
+    "nbsphinx",
+    "sphinx.ext.intersphinx",
+    "myst_parser",
+    "sphinx_design",
+]
+autosummary_generate = True
+katex_prerender = True
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = []
+
+master_doc = "index"
+autodoc_typehints = "none"
+
+
+html_theme_options = {
+    "navbar_align": "content",
+    "navbar_start": ["navbar-logo"],
+    "navbar_end": ["theme-switcher", "navbar-icon-links"],
+    "collapse_navigation": True,
+    "secondary_sidebar_items": ["page-toc"],
+    "show_prev_next": False,
+    "back_to_top_button": False,
+    "pygments_light_style": "a11y-light",
+    "pygments_dark_style": "a11y-dark",
+    "icon_links": [
+        {
+            # Label for this link
+            "name": "GitHub",
+            # URL where the link will redirect
+            "url": "https://github.com/facebookresearch/DRTK",
+            # Icon class (if "type": "fontawesome"), or path to local image (if "type": "local")
+            "icon": "fa-brands fa-github fa-2xl",
+            # The type of image to be used (see below for details)
+            "type": "fontawesome",
+            "attributes": {},
+        }
+    ],
+    "logo": {
+        "text": "DRTK",
+    },
+}
+
+html_static_path = ["_static"]
+html_css_files = ["custom.css"]

docs/source/index.md

+---
+myst:
+  html_meta:
+    "description lang=en": |
+      Top-level documentation for DRTK.
+html_theme.sidebar_secondary.remove: true
+---
+
+# DRTK – Differentiable Rendering Toolkit
+
+DRTK is a Python package built on top of PyTorch, offering differentiable rasterization functionality.
+We focus on rasterization due to its speed, providing functionality which is typically common in real-time graphics, while leaving shading and lighting/material models for user implementation.
+
+Rasterization is a widely used rendering technique due to its speed and efficiency, especially for real-time applications.
+However, it presents challenges when used in differentiable rendering pipelines, particularly at visibility boundaries, where non-differentiable operations, such as discrete pixel coverage and occlusions, occur.
+DRTK implements a novel approach to computing gradients at these visibility discontinuities described in [**Rasterized Edge Gradients: Handling Discontinuities Differentiably**](https://arxiv.org/abs/2405.02508), overcoming the limitations inherent in rasterization’s fixed-grid structure and z-buffering.
+DRTK keeps the rasterization process intact and efficient while enabling gradient propagation through occlusion boundaries and geometry intersections.
+DRTK provides a set of differentiable components to build custom differentiable rendering pipelines, like the following:
+
+{bdg-primary-line}`model` → {bdg-primary}`transform` → {bdg-primary}`rasterize` → {bdg-primary}`render` → {bdg-primary}`interpolate` → {bdg-warning}`CUSTOM SHADING` → {bdg-primary-line}`rendered image` → {bdg-primary}`edge_grad` → {bdg-warning}`LOSS FUNCTION`
+
+* **transform**: Projects 3D vertex positions onto the image plane of the camera.
+* **rasterize**: Performs rasterization, mapping output pixels to triangles.
+* **render**: Computes depth and barycentric images.
+* **interpolate**: Interpolates arbitrary vertex attributes.
+* **edge_grad**: Computes gradients at discontinuities.
+* **CUSTOM SHADING** and **LOSS FUNCTION**: User-defined components for shading and loss calculation.
+
+## Hello Triangle!
+
+Here's a simple "Hello Triangle" example with DRTK:
+```python
+import drtk
+import torch as th
+from torchvision.utils import save_image  # to save images
+
+# create vertex buffer of shape [1 x n_vertices x 3], here for triangle `n_vertices` == 3
+v = th.as_tensor([[[0, 511, 1], [255, 0, 1], [511, 511, 1]]]).float().cuda()
+
+# create index buffer
+vi = th.as_tensor([[0, 1, 2]]).int().cuda()
+
+# rasterize
+index_img = drtk.rasterize(v, vi, height=512, width=512)
+
+# compute baricentrics
+_, bary = drtk.render(v, vi, index_img)
+
+# we won't do shading, we'll just save the baricentrics and filter out the empty region
+# which is marked with `-1` in `index_img`
+img = bary * (index_img != -1)
+
+save_image(img, "render.png")
+```
+
+![hello triangle](/_static/hellow_triangle.png)
+
+
+## Installation
+To build and install the DRTK package as a wheel:
+```
+pip install wheel
+python setup.py  bdist_wheel
+pip install dist/drtk-<wheel_name>.whl
+```
+
+For in-place builds, useful for development:
+```
+python setup.py build_ext --inplace -j 1
+```
+
+
+## Requirements
+Cure dependencies:
+* CUDA Toolkit
+* PyTorch (>= 2.1.0)
+* numpy
+
+Some examples and tests may also require:
+* torchvision
+* opencv-python
+
+## Contributing
+See the [CONTRIBUTING](https://github.com/facebookresearch/DRTK//blob/main/CONTRIBUTING.md) file for how to help out.
+
+## License
+DRTK is MIT licensed, as found in the [LICENSE](https://github.com/facebookresearch/DRTK//blob/main/LICENSE) file.
+
+## Citation
+When using DRTK in academic projects, please cite:
+```bibtex
+@article{pidhorskyi2024rasterized,
+  title={Rasterized Edge Gradients: Handling Discontinuities Differentiably},
+  author={Pidhorskyi, Stanislav and Simon, Tomas and Schwartz, Gabriel and Wen, He and Sheikh, Yaser and Saragih, Jason},
+  journal={arXiv preprint arXiv:2405.02508},
+  year={2024}
+}
+```
+
+```{toctree}
+:glob:
+:maxdepth: 2
+:hidden:
+
+installation/index
+```
+
+```{toctree}
+:glob:
+:maxdepth: 2
+:hidden:
+
+tutorials/index
+```
+
+```{toctree}
+:glob:
+:maxdepth: 2
+:hidden:
+
+api_reference/index
+```
+
+```{toctree}
+:hidden:
+
+GitHub <https://github.com/facebookresearch/DRTK>
+```

docs/source/installation/index.rst

+
+:github_url: https://github.com/facebookresearch/DRTK
+
+Installation
+===================================
+
+Currently, we do not provide pre-compiled binaries for DRTK.
+You will need to build the package from source. The current version of DRTK is |version|.
+
+Prerequisites
+^^^^^^^^^^^^^
+
+Before installing DRTK, ensure you have the following prerequisites installed:
+
+* PyTorch >= 2.1.0
+* CUDA Toolkit
+
+Additionally, we recommend installing the following packages to run tests and examples:
+
+* torchvision
+* opencv_python
+
+Installing DRTK from GitHub using pip
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The easiest way to install DRTK is by using pip with the GitHub repository directly:
+
+.. code-block:: shell
+
+    # To install latest
+    pip install git+https://github.com/facebookresearch/DRTK.git
+
+.. code-block:: shell
+
+    # To install stable
+    pip install git+https://github.com/facebookresearch/DRTK.git@stable
+
+.. warning::
+
+    It may take significant amount of time to compile. The time could be 30 minutes or more.
+
+This should be enough in most cases, given that PyTorch, CUDA Toolkit, and Build Essentials for your platform
+are installed and the environment is correctly configured.
+
+
+.. _Specifying Architectures:
+
+Specifying Architectures
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If you know the CUDA architecture of the device where the code will run, then it would be better to specify it directly, e.g.:
+
+.. code-block:: shell
+
+    # TORCH_CUDA_ARCH_LIST can use "named" architecture, see table below
+    TORCH_CUDA_ARCH_LIST="Ampere" install git+https://github.com/facebookresearch/DRTK.git
+
+or specify numerical values for architectures explicitly:
+
+.. code-block:: shell
+
+    # TORCH_CUDA_ARCH_LIST can combine several architectures separated with semicolon or space.
+    # Add `+PTX` if you want also to save intermediate byte code for better compatibility.
+    TORCH_CUDA_ARCH_LIST="8.0;8.6+PTX" install git+https://github.com/facebookresearch/DRTK.git
+
+If ``TORCH_CUDA_ARCH_LIST`` is not specified, DRTK will build for the following architectures by default: 7.2, 7.5, 8.0, 8.6, 9.0.
+
+``TORCH_CUDA_ARCH_LIST`` can take one or more values from the supported named or numerical architectures list.
+When combining values, use a semicolon `;` or space to combine numerical values and the `+` symbol to combine named values.
+
+List of numerical architectures values supported by PyTorch: ``'3.5', '3.7', '5.0', '5.2', '5.3', '6.0', '6.1', '6.2', '7.0', '7.2', '7.5', '8.0', '8.6', '8.7', '8.9', '9.0', '9.0a'``.
+
+The "named" architectures supported by PyTorch are listed in the table below:
+
+.. list-table:: Named architectures
+   :header-rows: 1
+
+   * - Name
+     - Arch
+   * - Kepler+Tesla
+     - 3.7
+   * - Kepler
+     - 3.5+PTX
+   * - Maxwell+Tegra
+     - 5.3
+   * - Maxwell
+     - 5.0;5.2+PTX
+   * - Pascal
+     - 6.0;6.1+PTX
+   * - Volta+Tegra
+     - 7.2
+   * - Volta
+     - 7.0+PTX
+   * - Turing
+     - 7.5+PTX
+   * - Ampere+Tegra
+     - 8.7
+   * - Ampere
+     - 8.0;8.6+PTX
+   * - Ada
+     - 8.9+PTX
+   * - Hopper
+     - 9.0+PTX
+
+.. note::
+
+    We do not test the DRTK package on architectures before Volta.
+
+For more information about ``TORCH_CUDA_ARCH_LIST``, refer to the `PyTorch documentation <https://pytorch.org/docs/stable/cpp_extension.html#torch.utils.cpp_extension.CUDAExtension>`_ or view the `source code on GitHub <https://github.com/pytorch/pytorch/blob/c9653bf2ca6dd88b991d71abf836bd9a7a1d9dc3/torch/utils/cpp_extension.py#L1980>`_.
+
+Installing from a Cloned Repository
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Alternatively, you can install the package from a local repository clone.
+It could be helpful if you need to modify the package code.
+
+Clone the repository and ``cd`` into it:
+
+.. code-block:: shell
+
+    git clone https://github.com/facebookresearch/DRTK
+    cd DRTK
+
+Then, install the package from the path using ``pip``. Note the ``--no-build-isolation`` flag, it is needed for modern build
+system to disable building in a separate, clean Python environment.
+The reason is that it will install a default ``torch`` version from pip, which likely will not match the one already installed in the system (due to usage of ``--index-url``).
+
+.. code-block:: shell
+
+    pip install . --no-build-isolation
+
+
+Building and installing a wheel
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To build a wheel run:
+
+.. code-block:: shell
+
+    # You might need to install `build` first
+    # pip install --upgrade build
+    python -m build --wheel --no-isolation
+
+Alternatively, you can use the deprecated CLI of ``setuptools``:
+
+.. code-block:: shell
+
+    # You might need to install `wheel` first, though newer versions of setuptools do not require it anymore.
+    # pip install --upgrade wheel
+    python setup.py bdist_wheel
+
+Then, you will find a wheel in the ``dist/`` folder. You can install this wheel by running:
+
+.. code-block:: shell
+
+    pip install dist/drtk-<tags>.whl
+
+where ``<tags>`` are compatibility tags. You can figure them out by listing the ``dist/`` directory. E.g.:
+
+.. code-block:: shell
+
+    pip install dist/drtk-0.1.0-cp310-cp310-linux_x86_64.whl
+
+Reinstalling the Wheel
+^^^^^^^^^^^^^^^^^^^^^^
+
+If you have already installed the package using ``pip``, it will not reinstall the package unless the version number has been incremented.
+This behavior can be problematic if you are modifying the package locally and need to reinstall it.
+
+To force a reinstall, add the following arguments: ``--upgrade --force-reinstall --no-deps``. For example:
+
+.. code-block:: shell
+
+    pip install --upgrade --force-reinstall --no-deps .
+
+
+In place build
+^^^^^^^^^^^^^^^^^^^
+
+For package development, it can be beneficial to do an inplace build with:
+
+.. code-block:: shell
+
+    # There can be issues with concurrent build jobs, it is safer to specify `-inplace -j 1`
+    python setup.py build_ext --inplace -j 1
+
+Then you can use the root of the cloned repository as a working directory, and you should be able to do ``import drtk`` and run tests and examples.
+
+
+Troubleshooting
+================
+
+1. CUDA Error: No Kernel Image Available
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+**Example Error Message:**
+
+    RuntimeError: CUDA error: no kernel image is available for execution on the device
+
+
+**Cause:** This error occurs when the CUDA code was not built for the architecture of the device on which the code is running.
+
+**Solution:** Specify the correct architecture using the ``TORCH_CUDA_ARCH_LIST`` environment variable when building the package. Refer to the examples in the :ref:`Specifying Architectures` section above.
+
+2. Import Error: ``*.so`` Not Found
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+**Example Error Message:**
+
+    ImportError: libcudnn.so.9: cannot open shared object file: No such file or directory
+
+**Cause:** This issue is likely due to build isolation. Since DRTK currently does not distribute pre-compiled binaries, it is hard to get version mismatch otherwise.
+
+**Solution:** Ensure you include the ``--no-build-isolation`` flag when installing from a local clone to use the correct CUDA and PyTorch libraries from your current environment:
+
+.. code-block:: shell
+
+    python -m build --wheel --no-isolation
+
+and
+
+.. code-block:: shell
+
+    pip install . --no-build-isolation
+
+
+3. Compilation Errors in CUDA or PyTorch headers
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+**Example Error Message:**
+
+    error: no suitable conversion function from "const __half" to "unsigned short" exists
+
+**Cause:** This error typically indicates a compiler version mismatch. It is likely that your C++ or CUDA compiler version is too old to support some of the features.
+
+**Solution:** Please consult PyTorch and CUDA documentation to figure out what CUDA version is supported by your PyTorch version and what C++ compiler version is needed.
+
+
+4. C++ Compilation Errors in PyTorch header aten/src/ATen/core/boxing/impl/boxing.h
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+**Example Error Message:**
+
+    aten/src/ATen/core/boxing/impl/boxing.h:41:105: error: expected primary-expression before ‘>’ token
+
+
+**Cause:** This issue is related to problematic SFINAE logic in template code. It has been observed in some recent versions of PyTorch.
+
+**Solution:** One recommended solution is to add ``-std=c++20`` to the **nvcc** arguments, as suggested in `this GitHub issue <https://github.com/pytorch/pytorch/issues/122169>`_.
+This line has already been added to ``setup.py```:
+
+.. code-block:: python
+
+    nvcc_args.append("-std=c++20")
+
+
+.. warning::
+
+    If you are using an older version of the CUDA Toolkit, adding this flag might result in an error:
+
+    .. code-block:: shell
+
+        unrecognized command line option '-std=c++20'
+
+
+    In this case, you may need to remove the ``-std=c++20`` flag from ``setup.py``, or update your CUDA Toolkit to a version that supports C++20.
+
+.. note::
+
+    According to the comments in the `GitHub issue <https://github.com/pytorch/pytorch/issues/122169>`_, this was fixed in PyTorch v2.4.0 release.

docs/source/tutorials/index.rst

+
+:github_url: https://github.com/facebookresearch/DRTK
+
+Tutorials
+===================================
+
+.. toctree::
+   :maxdepth: 4
+   :caption: Getting Started
+   :hidden:
+
+   DRTK_Tutorial_1_hello_triangle
+   DRTK_Tutorial_2_optimizing_geometry
+
+.. grid::
+
+    .. grid-item-card:: :doc:`DRTK_Tutorial_1_hello_triangle`
+
+        In this tutorial, we will create a simple "hello triangle," which is a "hello world" of graphics programming, using DRTK.
+
+        .. image:: ../_static/tutorial_1_hello_triangle.png
+          :width: 400
+
+    .. grid-item-card:: :doc:`DRTK_Tutorial_2_optimizing_geometry`
+
+        In this tutorial, we will see how to setup a minimal example of geomtry optimization.
+
+        .. image:: ../_static/tutorial_2_animation.gif
+          :width: 400

drtk/mipmap_grid_sample.py

@@ -67,6 +67,7 @@ def mipmap_grid_sample(
             pyramid to be present. Such relaxed requirement leads to ambiguity when it needs to sample from
             the missing layer. The flag clip_grad only impacts the cases when needed layers from the mipmap
             pyramid are missing. In this scenario:
+
             - when False: it will sample from the last available layer. This will lead to aliasing and
               sparsely placed taps. The downside is that it may sample from arbitrary far regions of
               the texture.
@@ -137,6 +138,7 @@ def mipmap_grid_sample_ref(
     """
     A reference implementation for `mipmap_grid_sample`. See the doc string from `mipmap_grid_sample`
     The CUDA version of `mipmap_grid_sample` should behave the same as this referense implementation when:
+
     - `force_max_aniso` argument of `mipmap_grid_sample` is set to True
     - `clip_grad` argument of `mipmap_grid_sample` is set to False
     - `high_quality` argument of `mipmap_grid_sample_ref` is set to False