Docs (#36)

* update doc and bump version

* update docs with final scores

README.md

@@ -2,22 +2,23 @@
 [![Core Tests.](https://github.com/KAIR-BAIR/nerfacc/actions/workflows/code_checks.yml/badge.svg)](https://github.com/KAIR-BAIR/nerfacc/actions/workflows/code_checks.yml)
 [![Documentation Status](https://readthedocs.com/projects/plenoptix-nerfacc/badge/?version=latest)](https://plenoptix-nerfacc.readthedocs-hosted.com/en/latest/?badge=latest)
 
+
 This is a **tiny** toolbox  for **accelerating** NeRF training & rendering using PyTorch CUDA extensions. Plug-and-play for most of the NeRFs!
 
 ## Examples: 
 
 ``` bash
-# Instant-NGP NeRF
+# Instant-NGP Nerf
 python examples/train_ngp_nerf.py --train_split trainval --scene lego
 ```
 
 ``` bash
-# Vanilla MLP NeRF
+# Vanilla MLP Nerf
 python examples/train_mlp_nerf.py --train_split train --scene lego
 ```
 
 ```bash
-# MLP NeRF on Dynamic objects (D-NeRF)
+# MLP Nerf on Dynamic objects (D-Nerf)
 python examples/train_mlp_dnerf.py --train_split train --scene lego
 ```
 

docs/source/apis/grid.rst

+.. _`Occupancy Grid`:
+
 Occupancy Grid
 ===================================
 

docs/source/conf.py

@@ -8,7 +8,7 @@ project = "nerfacc"
 copyright = "2022, Ruilong"
 author = "Ruilong"
 
-release = "0.1"
+release = "0.1.1"
 version = "0.1.1"
 
 # -- General configuration

docs/source/examples/dnerf.rst

 Dynamic Scene
 ====================
 
+Here we trained a 8-layer-MLP for the radiance field and a 4-layer-MLP for the warping field,
+(similar to the T-Nerf model in the `D-Nerf`_ paper) on the `D-Nerf dataset`_. We used train 
+split for training and test split for evaluation. Our experiments are conducted on a 
+single NVIDIA TITAN RTX GPU. 
 
+Note:
+    The :ref:`Occupancy Grid` used in this example is shared by all the frames. In other words, 
+    instead of using it to indicate the opacity of an area at a single timestamp, 
+    Here we use it to indicate the `maximum` opacity at this area `over all the timestamps`.
+    It is not optimal but still makes the rendering very efficient.
 
 +----------------------+----------+---------+-------+---------+-------+--------+---------+-------+-------+
 |                      | bouncing | hell    | hook  | jumping | lego  | mutant | standup | trex  | AVG   |
 |                      | balls    | warrior |       | jacks   |       |        |         |       |       |
 +======================+==========+=========+=======+=========+=======+========+=========+=======+=======+
-| Paper (PSNR: ~2day)  | 38.93    | 25.02   | 29.25 | 32.80   | 21.64 | 31.29  | 32.79   | 31.75 | 30.43 |
+| D-Nerf (PSNR: ~2day) | 38.93    | 25.02   | 29.25 | 32.80   | 21.64 | 31.29  | 32.79   | 31.75 | 30.43 |
 +----------------------+----------+---------+-------+---------+-------+--------+---------+-------+-------+
 | Ours  (PSNR: ~50min) | 39.60    | 22.41   | 30.64 | 29.79   | 24.75 | 35.20  | 34.50   | 31.83 | 31.09 |
 +----------------------+----------+---------+-------+---------+-------+--------+---------+-------+-------+
 | Ours  (Training time)| 45min    | 49min   | 51min | 46min   | 53min | 57min  | 49min   | 46min | 50min |
 +----------------------+----------+---------+-------+---------+-------+--------+---------+-------+-------+
+
+.. _`D-Nerf`: https://arxiv.org/abs/2104.00677
+.. _`D-Nerf dataset`: https://www.dropbox.com/s/0bf6fl0ye2vz3vr/data.zip?dl=0

docs/source/examples/ngp.rst

+.. _`Instant-NGP Example`:
+
 Instant-NGP
 ====================
 
@@ -6,25 +8,27 @@ See code `examples/train_ngp_nerf.py` at our `github repository`_ for details.
 Benchmarks
 ------------
 
-Here we trained a NGP Nerf model on the NeRF-Synthetic dataset. We follow the same
-settings with the paper, which uses trainval split for training and test split for
+Here we trained a `Instant-NGP Nerf`_ model on the `Nerf-Synthetic dataset`_. We follow the same
+settings with the Instant-NGP paper, which uses trainval split for training and test split for
 evaluation. Our experiments are conducted on a single NVIDIA TITAN RTX GPU. The training
 memory footprint is about 3GB.
 
 .. note::
     
-    The paper makes use of the alpha channel in the images to apply random background
+    The Instant-NGP paper makes use of the alpha channel in the images to apply random background
     augmentation during training. Yet we only uses RGB values with a constant white background.
 
 +----------------------+-------+-------+------------+-------+--------+--------+--------+--------+--------+
 |                      | Lego  | Mic   | Materials  |Chair  |Hotdog  | Ficus  | Drums  | Ship   | AVG    |
 |                      |       |       |            |       |        |        |        |        |        |
 +======================+=======+=======+============+=======+========+========+========+========+========+
-| Paper (PSNR: 5min)   | 36.39 | 36.22 | 29.78      | 35.00 | 37.40  | 33.51  | 26.02  | 31.10  | 33.18  |
+|Instant-NGP(PSNR:5min)| 36.39 | 36.22 | 29.78      | 35.00 | 37.40  | 33.51  | 26.02  | 31.10  | 33.18  |
 +----------------------+-------+-------+------------+-------+--------+--------+--------+--------+--------+
 | Ours  (PSNR:4.5min)  | 36.71 | 36.78 | 29.06      | 36.10 | 37.88  | 32.07  | 25.83  | 31.39  | 33.23  |
 +----------------------+-------+-------+------------+-------+--------+--------+--------+--------+--------+
 | Ours  (Training time)| 286s  | 251s  | 250s       | 311s  | 275s   | 254s   | 249s   | 255s   | 266s   |
 +----------------------+-------+-------+------------+-------+--------+--------+--------+--------+--------+
 
-.. _`github repository`: : https://github.com/KAIR-BAIR/nerfacc/
\ No newline at end of file
+.. _`Instant-NGP Nerf`: https://arxiv.org/abs/2103.13497
+.. _`github repository`: https://github.com/KAIR-BAIR/nerfacc/
+.. _`Nerf-Synthetic dataset`: https://drive.google.com/drive/folders/1JDdLGDruGNXWnM1eqY1FNL9PlStjaKWi

docs/source/examples/unbounded.rst

 Unbounded Scene
 ====================
 
+Here we trained a `Instant-NGP Nerf`_  on the `MipNerf360`_ dataset. We used train 
+split for training and test split for evaluation. Our experiments are conducted on a 
+single NVIDIA TITAN RTX GPU. 
 
-+----------------------+-------+
-|                      | Garden| 
-|                      |       | 
-+======================+=======+
-| Paper (PSNR: ~ days) | 26.98 |
-+----------------------+-------+
-| Ours  (PSNR: ~ 1 hr) | 25.41 |
-+----------------------+-------+
-| Ours  (Training time)| 58min |
-+----------------------+-------+
+The main difference between working with unbounded scenes and bounded scenes, is that
+a contraction method is needed to map the infinite space to a finite :ref:`Occupancy Grid`.
+We have difference options provided for this (see :ref:`Occupancy Grid`). The experiments
+here is basically the Instant-NGP experiments (see :ref:`Instant-NGP Example`) with a contraction method
+that takes from `MipNerf360`_.
+
+.. note:: 
+    Even though we are comparing with `Nerf++`_ and `MipNerf360`_, the model and everything are
+    totally different with them. There are plenty of ideas from those papers that would be very
+    helpful for the performance, but we didn't adopt them. As this is just a simple example to 
+    show how to use the library, we didn't want to make it too complicated.
+
+
++----------------------+-------+-------+------------+-------+--------+--------+--------+
+|                      |Garden |Bicycle| Bonsai     |Counter|Kitchen | Room   | Stump  |
+|                      |       |       |            |       |        |        |        |
++======================+=======+=======+============+=======+========+========+========+
+|Nerf++(PSNR:~days)    | 24.32 | 22.64 | 29.15      | 26.38 | 27.80  | 28.87  | 24.34  |
++----------------------+-------+-------+------------+-------+--------+--------+--------+
+|MipNerf360(PSNR:~days)| 26.98 | 24.37 | 33.46      | 29.55 | 32.23  | 31.63  | 28.65  |
++----------------------+-------+-------+------------+-------+--------+--------+--------+
+| Ours  (PSNR:~1hr)    | 25.41 | 22.89 | 27.35      | 23.15 | 27.74  | 30.66  | 21.83  |
++----------------------+-------+-------+------------+-------+--------+--------+--------+
+| Ours  (Training time)| 40min | 35min | 47min      | 39min | 60min  | 41min  | 28min  |
++----------------------+-------+-------+------------+-------+--------+--------+--------+
+
+.. _`Instant-NGP Nerf`: https://arxiv.org/abs/2103.13497
+.. _`MipNerf360`: https://arxiv.org/abs/2111.12077
+.. _`Nerf++`: https://arxiv.org/abs/2010.07492

docs/source/examples/vanilla.rst

@@ -6,16 +6,27 @@ See code `examples/train_mlp_nerf.py` at our `github repository`_ for details.
 Benchmarks
 ------------
 
+Here we trained a 8-layer-MLP for the radiance field as in the `vanilla Nerf`_. We used the 
+train split for training and test split for evaluation as in the Nerf paper. Our experiments are 
+conducted on a single NVIDIA TITAN RTX GPU. 
+
+.. note:: 
+    The vanilla Nerf paper uses two MLPs for course-to-fine sampling. Instead here we only use a 
+    single MLP with more samples (1024). Both ways share the same spirit to do dense sampling 
+    around the surface. Our fast rendering inheritly skip samples away from the surface 
+    so we can simplly increase the number of samples with a single MLP, to achieve the same goal 
+    with the coarse-to-fine sampling, without runtime or memory issue.
 
 +----------------------+-------+-------+------------+-------+--------+--------+--------+--------+--------+
 |                      | Lego  | Mic   | Materials  |Chair  |Hotdog  | Ficus  | Drums  | Ship   | AVG    |
 |                      |       |       |            |       |        |        |        |        |        |
 +======================+=======+=======+============+=======+========+========+========+========+========+
-| Paper (PSNR: 1~2days)| 32.54 | 32.91 | 29.62      | 33.00 | 36.18  | 30.13  | 25.01  | 28.65  | 31.00  |
+| NeRF  (PSNR: ~days)  | 32.54 | 32.91 | 29.62      | 33.00 | 36.18  | 30.13  | 25.01  | 28.65  | 31.00  |
 +----------------------+-------+-------+------------+-------+--------+--------+--------+--------+--------+
 | Ours  (PSNR: ~50min) | 33.69 | 33.76 | 29.73      | 33.32 | 35.80  | 32.52  | 25.39  | 28.18  | 31.55  |
 +----------------------+-------+-------+------------+-------+--------+--------+--------+--------+--------+
 | Ours  (Training time)| 58min | 53min | 46min      | 62min | 56min  | 42min  | 52min  | 49min  | 52min  |
 +----------------------+-------+-------+------------+-------+--------+--------+--------+--------+--------+
 
-.. _`github repository`: : https://github.com/KAIR-BAIR/nerfacc/
\ No newline at end of file
+.. _`github repository`: : https://github.com/KAIR-BAIR/nerfacc/
+.. _`vanilla Nerf`: https://arxiv.org/abs/2003.08934

docs/source/index.rst

@@ -5,14 +5,13 @@ NerfAcc is a PyTorch Nerf acceleration toolbox for both training and inference.
 
 Using NerfAcc, 
 
-- The `vanilla Nerf model`_ with 8-layer MLPs can be trained to *better quality* (+~0.5 PNSR) \
+- The `vanilla Nerf`_ model with 8-layer MLPs can be trained to *better quality* (+~0.5 PNSR) \
   in *1 hour* rather than *1~2 days* as in the paper.
-- The `instant-ngp Nerf model`_ can be trained to *equal quality* with *9/10th* of the training time (4.5 minutes) \
+- The `Instant-NGP Nerf`_ model can be trained to *equal quality* with *9/10th* of the training time (4.5 minutes) \
   comparing to the official pure-CUDA implementation.
-- The `D-Nerf model`_ for *dynamic* objects can also be trained in *1 hour* \
+- The `D-Nerf`_ model for *dynamic* objects can also be trained in *1 hour* \
   rather than *2 days* as in the paper, and with *better quality* (+~0.5 PSNR).
-- *Unbounded scenes* from `MipNerf360`_ can also be trained in \
-  *~1 hour* and get comparable quality to the paper.
+- Both the *bounded* and *unbounded* scenes are supported.
 
 *And it is pure python interface with flexible apis!*
 
@@ -50,8 +49,8 @@ Installation:
    NeRFactory <https://plenoptix-nerfactory.readthedocs-hosted.com/>
 
 
-.. _`vanilla NeRF model`: https://arxiv.org/abs/2003.08934
-.. _`instant-ngp NeRF model`: https://arxiv.org/abs/2103.13497
-.. _`D-Nerf model`: https://arxiv.org/abs/2104.00677
+.. _`vanilla Nerf`: https://arxiv.org/abs/2003.08934
+.. _`Instant-NGP Nerf`: https://arxiv.org/abs/2103.13497
+.. _`D-Nerf`: https://arxiv.org/abs/2104.00677
 .. _`MipNerf360`: https://arxiv.org/abs/2111.12077
 .. _`pixel-Nerf`: https://arxiv.org/abs/2012.02190
\ No newline at end of file

examples/datasets/nerf_360_v2.py

@@ -53,7 +53,7 @@ def _load_colmap(root_fp: str, subject_id: str, split: str, factor: int = 1):
     # image names anymore.
     image_names = [imdata[k].name for k in imdata]
 
-    # # Switch from COLMAP (right, down, fwd) to NeRF (right, up, back) frame.
+    # # Switch from COLMAP (right, down, fwd) to Nerf (right, up, back) frame.
     # poses = poses @ np.diag([1, -1, -1, 1])
 
     # Get distortion parameters.
@@ -96,7 +96,7 @@ def _load_colmap(root_fp: str, subject_id: str, split: str, factor: int = 1):
 
     assert params is None, "Only support pinhole camera model."
 
-    # Previous NeRF results were generated with images sorted by filename,
+    # Previous Nerf results were generated with images sorted by filename,
     # ensure metrics are reported on the same test set.
     inds = np.argsort(image_names)
     image_names = [image_names[i] for i in inds]

examples/radiance_fields/mlp.py

@@ -163,7 +163,7 @@ class NerfMLP(nn.Module):
 
 
 class SinusoidalEncoder(nn.Module):
-    """Sinusoidal Positional Encoder used in NeRF."""
+    """Sinusoidal Positional Encoder used in Nerf."""
 
     def __init__(self, x_dim, min_deg, max_deg, use_identity: bool = True):
         super().__init__()

nerfacc/contraction.py

@@ -27,7 +27,7 @@ class ContractionType(Enum):
             .. math:: f(x) = \\frac{1}{2}(tanh(\\frac{x - x_0}{x_1 - x_0} - \\frac{1}{2}) + 1)
 
         UN_BOUNDED_SPHERE: Contract an unbounded space into a unit sphere. Used in
-            `Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields`_.
+            `Mip-Nerf 360: Unbounded Anti-Aliased Neural Radiance Fields`_.
 
             .. math:: 
                 f(x) = 
@@ -39,7 +39,7 @@ class ContractionType(Enum):
             .. math::
                 z(x) = \\frac{x - x_0}{x_1 - x_0} * 2 - 1
 
-            .. _Mip-NeRF 360\: Unbounded Anti-Aliased Neural Radiance Fields:
+            .. _Mip-Nerf 360\: Unbounded Anti-Aliased Neural Radiance Fields:
                 https://arxiv.org/abs/2111.12077
 
     """