README.md 9.62 KB
Newer Older
1
<img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/pytorch3dlogo.png" width="900"/>
facebook-github-bot's avatar
facebook-github-bot committed
2
3
4
5
6
7

[![CircleCI](https://circleci.com/gh/facebookresearch/pytorch3d.svg?style=svg)](https://circleci.com/gh/facebookresearch/pytorch3d)
[![Anaconda-Server Badge](https://anaconda.org/pytorch3d/pytorch3d/badges/version.svg)](https://anaconda.org/pytorch3d/pytorch3d)

# Introduction

8
PyTorch3D provides efficient, reusable components for 3D Computer Vision research with [PyTorch](https://pytorch.org).
facebook-github-bot's avatar
facebook-github-bot committed
9
10
11
12
13
14

Key features include:

- Data structure for storing and manipulating triangle meshes
- Efficient operations on triangle meshes (projective transformations, graph convolution, sampling, loss functions)
- A differentiable mesh renderer
15
- Implicitron, see [its README](projects/implicitron_trainer), a framework for new-view synthesis via implicit representations.
facebook-github-bot's avatar
facebook-github-bot committed
16

17
18
PyTorch3D is designed to integrate smoothly with deep learning methods for predicting and manipulating 3D data.
For this reason, all operators in PyTorch3D:
facebook-github-bot's avatar
facebook-github-bot committed
19
20
21
22
23
24

- Are implemented using PyTorch tensors
- Can handle minibatches of hetereogenous data
- Can be differentiated
- Can utilize GPUs for acceleration

25
Within FAIR, PyTorch3D has been used to power research projects such as [Mesh R-CNN](https://arxiv.org/abs/1906.02739).
facebook-github-bot's avatar
facebook-github-bot committed
26
27
28
29
30
31
32

## Installation

For detailed instructions refer to [INSTALL.md](INSTALL.md).

## License

Patrick Labatut's avatar
Patrick Labatut committed
33
PyTorch3D is released under the [BSD License](LICENSE).
facebook-github-bot's avatar
facebook-github-bot committed
34
35
36

## Tutorials

37
Get started with PyTorch3D by trying one of the tutorial notebooks.
facebook-github-bot's avatar
facebook-github-bot committed
38

39
|<img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/dolphin_deform.gif" width="310"/>|<img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/bundle_adjust.gif" width="310"/>|
facebook-github-bot's avatar
facebook-github-bot committed
40
|:-----------------------------------------------------------------------------------------------------------:|:--------------------------------------------------:|
41
| [Deform a sphere mesh to dolphin](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/deform_source_mesh_to_target_mesh.ipynb)| [Bundle adjustment](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/bundle_adjustment.ipynb) |
facebook-github-bot's avatar
facebook-github-bot committed
42

43
| <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/render_textured_mesh.gif" width="310"/> | <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/camera_position_teapot.gif" width="310" height="310"/>
facebook-github-bot's avatar
facebook-github-bot committed
44
|:------------------------------------------------------------:|:--------------------------------------------------:|
45
| [Render textured meshes](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/render_textured_meshes.ipynb)| [Camera position optimization](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/camera_position_optimization_with_differentiable_rendering.ipynb)|
facebook-github-bot's avatar
facebook-github-bot committed
46

47
| <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/pointcloud_render.png" width="310"/> | <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/cow_deform.gif" width="310" height="310"/>
48
|:------------------------------------------------------------:|:--------------------------------------------------:|
49
| [Render textured pointclouds](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/render_colored_points.ipynb)| [Fit a mesh with texture](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/fit_textured_mesh.ipynb)|
50

51
| <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/densepose_render.png" width="310"/> | <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/shapenet_render.png" width="310" height="310"/>
52
|:------------------------------------------------------------:|:--------------------------------------------------:|
53
| [Render DensePose data](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/render_densepose.ipynb)| [Load & Render ShapeNet data](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/dataloaders_ShapeNetCore_R2N2.ipynb)|
54

55
| <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/fit_textured_volume.gif" width="310"/> | <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/fit_nerf.gif" width="310" height="310"/>
56
|:------------------------------------------------------------:|:--------------------------------------------------:|
57
| [Fit Textured Volume](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/fit_textured_volume.ipynb)| [Fit A Simple Neural Radiance Field](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/fit_simple_neural_radiance_field.ipynb)|
58
59
60
61




facebook-github-bot's avatar
facebook-github-bot committed
62
63
## Documentation

64
Learn more about the API by reading the PyTorch3D [documentation](https://pytorch3d.readthedocs.org/).
facebook-github-bot's avatar
facebook-github-bot committed
65
66
67

We also have deep dive notes on several API components:

68
69
70
- [Heterogeneous Batching](https://github.com/facebookresearch/pytorch3d/tree/main/docs/notes/batching.md)
- [Mesh IO](https://github.com/facebookresearch/pytorch3d/tree/main/docs/notes/meshes_io.md)
- [Differentiable Rendering](https://github.com/facebookresearch/pytorch3d/tree/main/docs/notes/renderer_getting_started.md)
facebook-github-bot's avatar
facebook-github-bot committed
71

72
### Overview Video
73

74
We have created a short (~14 min) video tutorial providing an overview of the PyTorch3D codebase including several code examples. Click on the image below to watch the video on YouTube:
75

76
<a href="http://www.youtube.com/watch?v=Pph1r-x9nyY"><img src="http://img.youtube.com/vi/Pph1r-x9nyY/0.jpg" height="225" ></a>
77

facebook-github-bot's avatar
facebook-github-bot committed
78
79
## Development

80
We welcome new contributions to PyTorch3D and we will be actively maintaining this library! Please refer to [CONTRIBUTING.md](./.github/CONTRIBUTING.md) for full instructions on how to run the code, tests and linter, and submit your pull requests.
facebook-github-bot's avatar
facebook-github-bot committed
81

82
83
84
85
86
87
88
## Development and Compatibility

- `main` branch: actively developed, without any guarantee, Anything can be broken at any time
  - REMARK: this includes nightly builds which are built from `main`
  - HINT: the commit history can help locate regressions or changes
- backward-compatibility between releases: no guarantee. Best efforts to communicate breaking changes and facilitate migration of code or data (incl. models).

facebook-github-bot's avatar
facebook-github-bot committed
89
90
## Contributors

91
PyTorch3D is written and maintained by the Facebook AI Research Computer Vision Team.
facebook-github-bot's avatar
facebook-github-bot committed
92

93
94
95
In alphabetical order:

* Amitav Baruah
Nikhila Ravi's avatar
Nikhila Ravi committed
96
* Steve Branson
97
* Krzysztof Chalupka
98
99
100
101
* Luya Gao
* Georgia Gkioxari
* Taylor Gordon
* Justin Johnson
102
* Patrick Labatut
103
104
105
106
107
108
109
110
111
* Christoph Lassner
* Wan-Yen Lo
* David Novotny
* Nikhila Ravi
* Jeremy Reizenstein
* Dave Schnizlein
* Roman Shapovalov
* Olivia Wiles

facebook-github-bot's avatar
facebook-github-bot committed
112
113
## Citation

114
If you find PyTorch3D useful in your research, please cite our tech report:
facebook-github-bot's avatar
facebook-github-bot committed
115
116

```bibtex
117
118
@article{ravi2020pytorch3d,
    author = {Nikhila Ravi and Jeremy Reizenstein and David Novotny and Taylor Gordon
facebook-github-bot's avatar
facebook-github-bot committed
119
                  and Wan-Yen Lo and Justin Johnson and Georgia Gkioxari},
120
121
122
    title = {Accelerating 3D Deep Learning with PyTorch3D},
    journal = {arXiv:2007.08501},
    year = {2020},
facebook-github-bot's avatar
facebook-github-bot committed
123
124
}
```
Nikhila Ravi's avatar
Nikhila Ravi committed
125

Christoph Lassner's avatar
Christoph Lassner committed
126
127
128
129
If you are using the pulsar backend for sphere-rendering (the `PulsarPointRenderer` or `pytorch3d.renderer.points.pulsar.Renderer`), please cite the tech report:

```bibtex
@article{lassner2020pulsar,
130
131
    author = {Christoph Lassner and Michael Zollh\"ofer},
    title = {Pulsar: Efficient Sphere-based Neural Rendering},
Christoph Lassner's avatar
Christoph Lassner committed
132
133
134
135
    journal = {arXiv:2004.07484},
    year = {2020},
}
```
136

Nikhila Ravi's avatar
Nikhila Ravi committed
137
138
139
140
## News

Please see below for a timeline of the codebase updates in reverse chronological order. We are sharing updates on the releases as well as research projects which are built with PyTorch3D. The changelogs for the releases are available under [`Releases`](https://github.com/facebookresearch/pytorch3d/releases),  and the builds can be installed using `conda` as per the instructions in [INSTALL.md](INSTALL.md).

Jeremy Reizenstein's avatar
Jeremy Reizenstein committed
141
142
143
144
**[Aug 10th 2022]:**   PyTorch3D [v0.7.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.7.0) released with Implicitron and MeshRasterizerOpenGL.

**[Apr 28th 2022]:**   PyTorch3D [v0.6.2](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.6.2) released

Jeremy Reizenstein's avatar
Jeremy Reizenstein committed
145
146
**[Dec 16th 2021]:**   PyTorch3D [v0.6.1](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.6.1) released

Jeremy Reizenstein's avatar
Jeremy Reizenstein committed
147
148
149
150
**[Oct 6th 2021]:**   PyTorch3D [v0.6.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.6.0) released

**[Aug 5th 2021]:**   PyTorch3D [v0.5.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.5.0) released

151
**[Feb 9th 2021]:** PyTorch3D [v0.4.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.4.0) released with support for implicit functions, volume rendering and a [reimplementation of NeRF](https://github.com/facebookresearch/pytorch3d/tree/main/projects/nerf).
Christoph Lassner's avatar
Christoph Lassner committed
152

Nikhila Ravi's avatar
Nikhila Ravi committed
153
154
155
**[November 2nd 2020]:** PyTorch3D [v0.3.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.3.0) released, integrating the pulsar backend.

**[Aug 28th 2020]:**   PyTorch3D [v0.2.5](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.2.5) released
156

157
158
**[July 17th 2020]:**   PyTorch3D tech report published on ArXiv: https://arxiv.org/abs/2007.08501

Nikhila Ravi's avatar
Nikhila Ravi committed
159
**[April 24th 2020]:**   PyTorch3D [v0.2.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.2.0) released
Nikhila Ravi's avatar
Nikhila Ravi committed
160
161
162

**[March 25th 2020]:**   [SynSin](https://arxiv.org/abs/1912.08804) codebase released using PyTorch3D: https://github.com/facebookresearch/synsin

Nikhila Ravi's avatar
Nikhila Ravi committed
163
**[March 8th 2020]:**   PyTorch3D [v0.1.1](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.1.1) bug fix release
Nikhila Ravi's avatar
Nikhila Ravi committed
164

Nikhila Ravi's avatar
Nikhila Ravi committed
165
**[Jan 23rd 2020]:**   PyTorch3D [v0.1.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.1.0) released. [Mesh R-CNN](https://arxiv.org/abs/1906.02739) codebase released: https://github.com/facebookresearch/meshrcnn