README.md 10.3 KB
Newer Older
1
<img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/pytorch3dlogo.png" width="900"/>
facebook-github-bot's avatar
facebook-github-bot committed
2
3
4
5
6
7

[![CircleCI](https://circleci.com/gh/facebookresearch/pytorch3d.svg?style=svg)](https://circleci.com/gh/facebookresearch/pytorch3d)
[![Anaconda-Server Badge](https://anaconda.org/pytorch3d/pytorch3d/badges/version.svg)](https://anaconda.org/pytorch3d/pytorch3d)

# Introduction

8
PyTorch3D provides efficient, reusable components for 3D Computer Vision research with [PyTorch](https://pytorch.org).
facebook-github-bot's avatar
facebook-github-bot committed
9
10
11
12
13
14

Key features include:

- Data structure for storing and manipulating triangle meshes
- Efficient operations on triangle meshes (projective transformations, graph convolution, sampling, loss functions)
- A differentiable mesh renderer
15
- Implicitron, see [its README](projects/implicitron_trainer), a framework for new-view synthesis via implicit representations.
facebook-github-bot's avatar
facebook-github-bot committed
16

17
18
PyTorch3D is designed to integrate smoothly with deep learning methods for predicting and manipulating 3D data.
For this reason, all operators in PyTorch3D:
facebook-github-bot's avatar
facebook-github-bot committed
19
20
21
22
23
24

- Are implemented using PyTorch tensors
- Can handle minibatches of hetereogenous data
- Can be differentiated
- Can utilize GPUs for acceleration

25
Within FAIR, PyTorch3D has been used to power research projects such as [Mesh R-CNN](https://arxiv.org/abs/1906.02739).
facebook-github-bot's avatar
facebook-github-bot committed
26
27
28
29
30
31
32

## Installation

For detailed instructions refer to [INSTALL.md](INSTALL.md).

## License

Patrick Labatut's avatar
Patrick Labatut committed
33
PyTorch3D is released under the [BSD License](LICENSE).
facebook-github-bot's avatar
facebook-github-bot committed
34
35
36

## Tutorials

37
Get started with PyTorch3D by trying one of the tutorial notebooks.
facebook-github-bot's avatar
facebook-github-bot committed
38

39
|<img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/dolphin_deform.gif" width="310"/>|<img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/bundle_adjust.gif" width="310"/>|
facebook-github-bot's avatar
facebook-github-bot committed
40
|:-----------------------------------------------------------------------------------------------------------:|:--------------------------------------------------:|
41
| [Deform a sphere mesh to dolphin](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/deform_source_mesh_to_target_mesh.ipynb)| [Bundle adjustment](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/bundle_adjustment.ipynb) |
facebook-github-bot's avatar
facebook-github-bot committed
42

43
| <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/render_textured_mesh.gif" width="310"/> | <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/camera_position_teapot.gif" width="310" height="310"/>
facebook-github-bot's avatar
facebook-github-bot committed
44
|:------------------------------------------------------------:|:--------------------------------------------------:|
45
| [Render textured meshes](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/render_textured_meshes.ipynb)| [Camera position optimization](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/camera_position_optimization_with_differentiable_rendering.ipynb)|
facebook-github-bot's avatar
facebook-github-bot committed
46

47
| <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/pointcloud_render.png" width="310"/> | <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/cow_deform.gif" width="310" height="310"/>
48
|:------------------------------------------------------------:|:--------------------------------------------------:|
49
| [Render textured pointclouds](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/render_colored_points.ipynb)| [Fit a mesh with texture](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/fit_textured_mesh.ipynb)|
50

51
| <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/densepose_render.png" width="310"/> | <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/shapenet_render.png" width="310" height="310"/>
52
|:------------------------------------------------------------:|:--------------------------------------------------:|
53
| [Render DensePose data](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/render_densepose.ipynb)| [Load & Render ShapeNet data](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/dataloaders_ShapeNetCore_R2N2.ipynb)|
54

55
| <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/fit_textured_volume.gif" width="310"/> | <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/fit_nerf.gif" width="310" height="310"/>
56
|:------------------------------------------------------------:|:--------------------------------------------------:|
57
| [Fit Textured Volume](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/fit_textured_volume.ipynb)| [Fit A Simple Neural Radiance Field](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/fit_simple_neural_radiance_field.ipynb)|
58

59
60
61
62
63
| <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/fit_textured_volume.gif" width="310"/> | <img src="https://raw.githubusercontent.com/facebookresearch/pytorch3d/main/.github/implicitron_config.gif" width="310" height="310"/>
|:------------------------------------------------------------:|:--------------------------------------------------:|
| [Fit Textured Volume in Implicitron](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/implicitron_volumes.ipynb)| [Implicitron Config System](https://github.com/facebookresearch/pytorch3d/blob/main/docs/tutorials/implicitron_config_system.ipynb)|


64
65
66



facebook-github-bot's avatar
facebook-github-bot committed
67
68
## Documentation

69
Learn more about the API by reading the PyTorch3D [documentation](https://pytorch3d.readthedocs.org/).
facebook-github-bot's avatar
facebook-github-bot committed
70
71
72

We also have deep dive notes on several API components:

73
74
75
- [Heterogeneous Batching](https://github.com/facebookresearch/pytorch3d/tree/main/docs/notes/batching.md)
- [Mesh IO](https://github.com/facebookresearch/pytorch3d/tree/main/docs/notes/meshes_io.md)
- [Differentiable Rendering](https://github.com/facebookresearch/pytorch3d/tree/main/docs/notes/renderer_getting_started.md)
facebook-github-bot's avatar
facebook-github-bot committed
76

77
### Overview Video
78

79
We have created a short (~14 min) video tutorial providing an overview of the PyTorch3D codebase including several code examples. Click on the image below to watch the video on YouTube:
80

81
<a href="http://www.youtube.com/watch?v=Pph1r-x9nyY"><img src="http://img.youtube.com/vi/Pph1r-x9nyY/0.jpg" height="225" ></a>
82

facebook-github-bot's avatar
facebook-github-bot committed
83
84
## Development

85
We welcome new contributions to PyTorch3D and we will be actively maintaining this library! Please refer to [CONTRIBUTING.md](./.github/CONTRIBUTING.md) for full instructions on how to run the code, tests and linter, and submit your pull requests.
facebook-github-bot's avatar
facebook-github-bot committed
86

87
88
89
90
91
92
93
## Development and Compatibility

- `main` branch: actively developed, without any guarantee, Anything can be broken at any time
  - REMARK: this includes nightly builds which are built from `main`
  - HINT: the commit history can help locate regressions or changes
- backward-compatibility between releases: no guarantee. Best efforts to communicate breaking changes and facilitate migration of code or data (incl. models).

facebook-github-bot's avatar
facebook-github-bot committed
94
95
## Contributors

96
PyTorch3D is written and maintained by the Facebook AI Research Computer Vision Team.
facebook-github-bot's avatar
facebook-github-bot committed
97

98
99
100
In alphabetical order:

* Amitav Baruah
Nikhila Ravi's avatar
Nikhila Ravi committed
101
* Steve Branson
102
* Krzysztof Chalupka
103
104
105
106
* Luya Gao
* Georgia Gkioxari
* Taylor Gordon
* Justin Johnson
107
* Patrick Labatut
108
109
110
111
112
113
114
115
116
* Christoph Lassner
* Wan-Yen Lo
* David Novotny
* Nikhila Ravi
* Jeremy Reizenstein
* Dave Schnizlein
* Roman Shapovalov
* Olivia Wiles

facebook-github-bot's avatar
facebook-github-bot committed
117
118
## Citation

119
If you find PyTorch3D useful in your research, please cite our tech report:
facebook-github-bot's avatar
facebook-github-bot committed
120
121

```bibtex
122
123
@article{ravi2020pytorch3d,
    author = {Nikhila Ravi and Jeremy Reizenstein and David Novotny and Taylor Gordon
facebook-github-bot's avatar
facebook-github-bot committed
124
                  and Wan-Yen Lo and Justin Johnson and Georgia Gkioxari},
125
126
127
    title = {Accelerating 3D Deep Learning with PyTorch3D},
    journal = {arXiv:2007.08501},
    year = {2020},
facebook-github-bot's avatar
facebook-github-bot committed
128
129
}
```
Nikhila Ravi's avatar
Nikhila Ravi committed
130

Christoph Lassner's avatar
Christoph Lassner committed
131
132
133
134
If you are using the pulsar backend for sphere-rendering (the `PulsarPointRenderer` or `pytorch3d.renderer.points.pulsar.Renderer`), please cite the tech report:

```bibtex
@article{lassner2020pulsar,
135
136
    author = {Christoph Lassner and Michael Zollh\"ofer},
    title = {Pulsar: Efficient Sphere-based Neural Rendering},
Christoph Lassner's avatar
Christoph Lassner committed
137
138
139
140
    journal = {arXiv:2004.07484},
    year = {2020},
}
```
141

Nikhila Ravi's avatar
Nikhila Ravi committed
142
143
144
145
## News

Please see below for a timeline of the codebase updates in reverse chronological order. We are sharing updates on the releases as well as research projects which are built with PyTorch3D. The changelogs for the releases are available under [`Releases`](https://github.com/facebookresearch/pytorch3d/releases),  and the builds can be installed using `conda` as per the instructions in [INSTALL.md](INSTALL.md).

Jeremy Reizenstein's avatar
Jeremy Reizenstein committed
146
147
148
149
**[Aug 10th 2022]:**   PyTorch3D [v0.7.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.7.0) released with Implicitron and MeshRasterizerOpenGL.

**[Apr 28th 2022]:**   PyTorch3D [v0.6.2](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.6.2) released

Jeremy Reizenstein's avatar
Jeremy Reizenstein committed
150
151
**[Dec 16th 2021]:**   PyTorch3D [v0.6.1](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.6.1) released

Jeremy Reizenstein's avatar
Jeremy Reizenstein committed
152
153
154
155
**[Oct 6th 2021]:**   PyTorch3D [v0.6.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.6.0) released

**[Aug 5th 2021]:**   PyTorch3D [v0.5.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.5.0) released

156
**[Feb 9th 2021]:** PyTorch3D [v0.4.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.4.0) released with support for implicit functions, volume rendering and a [reimplementation of NeRF](https://github.com/facebookresearch/pytorch3d/tree/main/projects/nerf).
Christoph Lassner's avatar
Christoph Lassner committed
157

Nikhila Ravi's avatar
Nikhila Ravi committed
158
159
160
**[November 2nd 2020]:** PyTorch3D [v0.3.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.3.0) released, integrating the pulsar backend.

**[Aug 28th 2020]:**   PyTorch3D [v0.2.5](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.2.5) released
161

162
163
**[July 17th 2020]:**   PyTorch3D tech report published on ArXiv: https://arxiv.org/abs/2007.08501

Nikhila Ravi's avatar
Nikhila Ravi committed
164
**[April 24th 2020]:**   PyTorch3D [v0.2.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.2.0) released
Nikhila Ravi's avatar
Nikhila Ravi committed
165
166
167

**[March 25th 2020]:**   [SynSin](https://arxiv.org/abs/1912.08804) codebase released using PyTorch3D: https://github.com/facebookresearch/synsin

Nikhila Ravi's avatar
Nikhila Ravi committed
168
**[March 8th 2020]:**   PyTorch3D [v0.1.1](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.1.1) bug fix release
Nikhila Ravi's avatar
Nikhila Ravi committed
169

Nikhila Ravi's avatar
Nikhila Ravi committed
170
**[Jan 23rd 2020]:**   PyTorch3D [v0.1.0](https://github.com/facebookresearch/pytorch3d/releases/tag/v0.1.0) released. [Mesh R-CNN](https://arxiv.org/abs/1906.02739) codebase released: https://github.com/facebookresearch/meshrcnn