@@ -513,7 +513,7 @@ Rendered in 4×4 higher resolution and downsampled
...
@@ -513,7 +513,7 @@ Rendered in 4×4 higher resolution and downsampled
<p>Nvdiffrast follows OpenGL's coordinate systems and other conventions. This is partially because we use OpenGL to accelerate the rasterization operation, but mostly so that there is a <ahref="https://xkcd.com/927/">single standard to follow</a>.</p>
<p>Nvdiffrast follows OpenGL's coordinate systems and other conventions. This is partially because we use OpenGL to accelerate the rasterization operation, but mostly so that there is a <ahref="https://xkcd.com/927/">single standard to follow</a>.</p>
<ul>
<ul>
<li>
<li>
When rasterizing, the normalized device coordinates — i.e., clip-space coordinates after division by <spanclass="math inline"><em>w</em></span> — map to screen so that<spanclass="math inline"><em>x</em></span> increases towards right side of screen,<spanclass="math inline"><em>y</em></span> increases towards top of screen, and <strong><spanclass="math inline"><em>z</em></span> increases towards the viewer</strong>.
In OpenGL convention, the perspective projection matrix (as implemented in, e.g., <ahref="https://github.com/NVlabs/nvdiffrast/blob/main/samples/torch/util.py#L16-L20"><code>utils.projection()</code></a> in our samples and <ahref="https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glFrustum.xml"><code>glFrustum()</code></a> in OpenGL) treats the view-space<spanclass="math inline"><em>z</em></span>as increasing towards the viewer. However, <em>after</em> multiplication by perspective projection matrix, the homogeneous <ahref="https://en.wikipedia.org/wiki/Clip_coordinates">clip-space</a> coordinate<spanclass="math inline"><em>z</em></span>/<spanclass="math inline"><em>w</em></span> increases away from the viewer. Hence, a larger depth value in the rasterizer output tensor also corresponds to a surface further away from the viewer.
</li>
</li>
<li>
<li>
<strong>The memory order of image data in OpenGL, and consequently in nvdiffrast, is bottom-up.</strong> This means that row 0 of a tensor containing an image is the bottom row of the texture/image, which is the opposite of the more common scanline order. If you want to keep your image data in the conventional top-down order in your code, but have it logically the right way up inside nvdiffrast, you will need to flip the images vertically when crossing the boundary.
<strong>The memory order of image data in OpenGL, and consequently in nvdiffrast, is bottom-up.</strong> This means that row 0 of a tensor containing an image is the bottom row of the texture/image, which is the opposite of the more common scanline order. If you want to keep your image data in the conventional top-down order in your code, but have it logically the right way up inside nvdiffrast, you will need to flip the images vertically when crossing the boundary.
...
@@ -782,6 +782,7 @@ Third depth layer
...
@@ -782,6 +782,7 @@ Third depth layer
<p>In manual mode, the user assumes the responsibility of setting and releasing the OpenGL context. Most of the time, if you don't have any other libraries that would be using OpenGL, you can just set the context once after having created it and keep it set until the program exits. However, keep in mind that the active OpenGL context is a thread-local resource, so it needs to be set in the same CPU thread as it will be used, and it cannot be set simultaneously in multiple CPU threads.</p>
<p>In manual mode, the user assumes the responsibility of setting and releasing the OpenGL context. Most of the time, if you don't have any other libraries that would be using OpenGL, you can just set the context once after having created it and keep it set until the program exits. However, keep in mind that the active OpenGL context is a thread-local resource, so it needs to be set in the same CPU thread as it will be used, and it cannot be set simultaneously in multiple CPU threads.</p>
<h2id="samples">Samples</h2>
<h2id="samples">Samples</h2>
<p>Nvdiffrast comes with a set of samples that were crafted to support the research paper. Each sample is available in both PyTorch and TensorFlow versions. Details such as command-line parameters, logging format, etc., may not be identical between the versions, and generally the PyTorch versions should be considered definitive. The command-line examples below are for the PyTorch versions.</p>
<p>Nvdiffrast comes with a set of samples that were crafted to support the research paper. Each sample is available in both PyTorch and TensorFlow versions. Details such as command-line parameters, logging format, etc., may not be identical between the versions, and generally the PyTorch versions should be considered definitive. The command-line examples below are for the PyTorch versions.</p>
<p>Enabling interactive display using the <code>--display-interval</code> parameter works on Windows but is likely to fail on Linux. Our Dockerfile is set up to support headless rendering only, and thus cannot show an interactive result window.</p>
<p>This is a minimal sample that renders a triangle and saves the resulting image into a file (<code>tri.png</code>) in the current directory. Running this should be the first step to verify that you have everything set up correctly. Rendering is done using the rasterization and interpolation operations, so getting the correct output image means that both OpenGL and CUDA are working as intended under the hood.</p>
<p>This is a minimal sample that renders a triangle and saves the resulting image into a file (<code>tri.png</code>) in the current directory. Running this should be the first step to verify that you have everything set up correctly. Rendering is done using the rasterization and interpolation operations, so getting the correct output image means that both OpenGL and CUDA are working as intended under the hood.</p>
<p>Example command line:</p>
<p>Example command line:</p>
...
@@ -895,9 +896,16 @@ Interactive view of pose.py
...
@@ -895,9 +896,16 @@ Interactive view of pose.py
<h2id="pytorch-api-reference">PyTorch API reference</h2>
<h2id="pytorch-api-reference">PyTorch API reference</h2>
<pclass="shortdesc">Create a new OpenGL rasterizer context.</p><pclass="longdesc">Creating an OpenGL context is a slow operation so you should reuse the same
<pclass="shortdesc">Create a new OpenGL rasterizer context.</p><pclass="longdesc">Creating an OpenGL context is a slow operation so you should usually reuse the same
context in all calls to <code>rasterize()</code> on the same CPU thread. The OpenGL context
context in all calls to <code>rasterize()</code> on the same CPU thread. The OpenGL context
is deleted when the object is destroyed.</p><divclass="arguments">Arguments:</div><tableclass="args"><trclass="arg"><tdclass="argname">output_db</td><tdclass="arg_short">Compute and output image-space derivates of barycentrics.</td></tr><trclass="arg"><tdclass="argname">mode</td><tdclass="arg_short">OpenGL context handling mode. Valid values are 'manual' and 'automatic'.</td></tr><trclass="arg"><tdclass="argname">device</td><tdclass="arg_short">Cuda device on which the context is created. Type can be
is deleted when the object is destroyed.</p><pclass="longdesc">Side note: When using the OpenGL context in a rasterization operation, the
context's internal framebuffer object is automatically enlarged to accommodate the
rasterization operation's output shape, but it is never shrunk in size until the
context is destroyed. Thus, if you need to rasterize, say, deep low-resolution
tensors and also shallow high-resolution tensors, you can conserve GPU memory by
creating two separate OpenGL contexts for these tasks. In this scenario, using the
same OpenGL context for both tasks would end up reserving GPU memory for a deep,
high-resolution output tensor.</p><divclass="arguments">Arguments:</div><tableclass="args"><trclass="arg"><tdclass="argname">output_db</td><tdclass="arg_short">Compute and output image-space derivates of barycentrics.</td></tr><trclass="arg"><tdclass="argname">mode</td><tdclass="arg_short">OpenGL context handling mode. Valid values are 'manual' and 'automatic'.</td></tr><trclass="arg"><tdclass="argname">device</td><tdclass="arg_short">Cuda device on which the context is created. Type can be
<code>torch.device</code>, string (e.g., <code>'cuda:1'</code>), or int. If not
<code>torch.device</code>, string (e.g., <code>'cuda:1'</code>), or int. If not
specified, context will be created on currently active Cuda
specified, context will be created on currently active Cuda
device.</td></tr></table><divclass="methods">Methods, only available if context was created in manual mode:</div><tableclass="args"><trclass="arg"><tdclass="argname">set_context()</td><tdclass="arg_short">Set (activate) OpenGL context in the current CPU thread.</td></tr><trclass="arg"><tdclass="argname">release_context()</td><tdclass="arg_short">Release (deactivate) currently active OpenGL context.</td></tr></table><divclass="returns">Returns:<divclass="return_description">The newly created OpenGL rasterizer context.</div></div></div>
device.</td></tr></table><divclass="methods">Methods, only available if context was created in manual mode:</div><tableclass="args"><trclass="arg"><tdclass="argname">set_context()</td><tdclass="arg_short">Set (activate) OpenGL context in the current CPU thread.</td></tr><trclass="arg"><tdclass="argname">release_context()</td><tdclass="arg_short">Release (deactivate) currently active OpenGL context.</td></tr></table><divclass="returns">Returns:<divclass="return_description">The newly created OpenGL rasterizer context.</div></div></div>
...
@@ -1002,9 +1010,9 @@ severity will be silent.</td></tr></table></div>
...
@@ -1002,9 +1010,9 @@ severity will be silent.</td></tr></table></div>
<p>This work is made available under the <ahref="https://github.com/NVlabs/nvdiffrast/blob/main/LICENSE.txt">Nvidia Source Code License</a>.</p>
<p>This work is made available under the <ahref="https://github.com/NVlabs/nvdiffrast/blob/main/LICENSE.txt">Nvidia Source Code License</a>.</p>
<p>For business inquiries, please contact <ahref="mailto:researchinquiries@nvidia.com">researchinquiries@nvidia.com</a></p>
<p>For business inquiries, please visit our website and submit the form: <ahref="https://www.nvidia.com/en-us/research/inquiries/">NVIDIA Research Licensing</a></p>
<p>We do not currently accept outside contributions in the form of pull requests.</p>
<p>We do not currently accept outside contributions in the form of pull requests.</p>
<p>Environment map stored as part of <code>samples/data/envphong.npz</code> is derived from a Wave Engine <ahref="https://github.com/WaveEngine/Samples/tree/master/Materials/EnvironmentMap/Content/Assets/CubeMap.cubemap">sample material</a> originally shared under <ahref="https://github.com/WaveEngine/Samples/blob/master/LICENSE.md">MIT License</a>. Mesh and texture stored as part of <code>samples/data/earth.npz</code> are derived from <ahref="https://www.turbosquid.com/3d-models/3d-realistic-earth-photorealistic-2k-1279125">3D Earth Photorealistic 2K</a> model originally made available under <ahref="https://blog.turbosquid.com/turbosquid-3d-model-license/#3d-model-license">TurboSquid 3D Model License</a>.</p>
<p>Environment map stored as part of <code>samples/data/envphong.npz</code> is derived from a Wave Engine <ahref="https://github.com/WaveEngine/Samples-2.5/tree/master/Materials/EnvironmentMap/Content/Assets/CubeMap.cubemap">sample material</a> originally shared under <ahref="https://github.com/WaveEngine/Samples-2.5/blob/master/LICENSE.md">MIT License</a>. Mesh and texture stored as part of <code>samples/data/earth.npz</code> are derived from <ahref="https://www.turbosquid.com/3d-models/3d-realistic-earth-photorealistic-2k-1279125">3D Earth Photorealistic 2K</a> model originally made available under <ahref="https://blog.turbosquid.com/turbosquid-3d-model-license/#3d-model-license">TurboSquid 3D Model License</a>.</p>
<h2id="citation">Citation</h2>
<h2id="citation">Citation</h2>
<pre><code>@article{Laine2020diffrast,
<pre><code>@article{Laine2020diffrast,
title = {Modular Primitives for High-Performance Differentiable Rendering},
title = {Modular Primitives for High-Performance Differentiable Rendering},
