README.md 9.87 KB
Newer Older
Soumith Chintala's avatar
Soumith Chintala committed
1
torchaudio: an audio library for PyTorch
Vincent QB's avatar
Vincent QB committed
2
========================================
Soumith Chintala's avatar
Soumith Chintala committed
3

moto's avatar
moto committed
4
5
[![Build Status](https://circleci.com/gh/pytorch/audio.svg?style=svg)](https://app.circleci.com/pipelines/github/pytorch/audio)
[![Documentation](https://img.shields.io/badge/dynamic/json.svg?label=docs&url=https%3A%2F%2Fpypi.org%2Fpypi%2Ftorchaudio%2Fjson&query=%24.info.version&colorB=brightgreen&prefix=v)](https://pytorch.org/audio/)
6

jamarshon's avatar
jamarshon committed
7
The aim of torchaudio is to apply [PyTorch](https://github.com/pytorch/pytorch) to
8
the audio domain. By supporting PyTorch, torchaudio follows the same philosophy
jamarshon's avatar
jamarshon committed
9
10
of providing strong GPU acceleration, having a focus on trainable features through
the autograd system, and having consistent style (tensor names and dimension names).
11
Therefore, it is primarily a machine learning library and not a general signal
12
processing library. The benefits of PyTorch can be seen in torchaudio through
13
having all the computations be through PyTorch operations which makes it easy
jamarshon's avatar
jamarshon committed
14
15
to use and feel like a natural extension.

16
- [Support audio I/O (Load files, Save files)](http://pytorch.org/audio/stable/)
17
  - Load the following formats into a torch Tensor using SoX
18
19
20
    - mp3, wav, aac, ogg, flac, avr, cdda, cvs/vms,
    - aiff, au, amr, mp2, mp4, ac3, avi, wmv,
    - mpeg, ircam and any other format supported by libsox.
21
22
    - [Kaldi (ark/scp)](http://pytorch.org/audio/stable/kaldi_io.html)
- [Dataloaders for common audio datasets (VCTK, YesNo)](http://pytorch.org/audio/stable/datasets.html)
23
- Common audio transforms
24
    - [Spectrogram, AmplitudeToDB, MelScale, MelSpectrogram, MFCC, MuLawEncoding, MuLawDecoding, Resample](http://pytorch.org/audio/stable/transforms.html)
25
- Compliance interfaces: Run code using PyTorch that align with other libraries
26
    - [Kaldi: spectrogram, fbank, mfcc, resample_waveform](https://pytorch.org/audio/stable/compliance.kaldi.html)
Soumith Chintala's avatar
Soumith Chintala committed
27
28
29

Dependencies
------------
moto's avatar
moto committed
30
* PyTorch (See below for the compatible versions)
31
* [optional] vesis84/kaldi-io-for-python commit cb46cb1f44318a5d04d4941cf39084c5b021241e or above
Soumith Chintala's avatar
Soumith Chintala committed
32

33
The following are the corresponding ``torchaudio`` versions and supported Python versions.
moto's avatar
moto committed
34
35
36

| ``torch``                | ``torchaudio``           | ``python``                      |
| ------------------------ | ------------------------ | ------------------------------- |
37
38
39
40
41
42
| ``master`` / ``nightly`` | ``master`` / ``nightly`` | ``>=3.6``, ``<=3.9``            |
| ``1.8.0``                | ``0.8.0``                | ``>=3.6``, ``<=3.9``            |
| ``1.7.1``                | ``0.7.2``                | ``>=3.6``, ``<=3.9``            |
| ``1.7.0``                | ``0.7.0``                | ``>=3.6``, ``<=3.8``            |
| ``1.6.0``                | ``0.6.0``                | ``>=3.6``, ``<=3.8``            |
| ``1.5.0``                | ``0.5.0``                | ``>=3.5``, ``<=3.8``            |
moto's avatar
moto committed
43
44
45
| ``1.4.0``                | ``0.4.0``                | ``==2.7``, ``>=3.5``, ``<=3.8`` |


Soumith Chintala's avatar
Soumith Chintala committed
46
47
48
Installation
------------

49
### Binary Distributions
jamarshon's avatar
jamarshon committed
50

51
To install the latest version using anaconda, run:
52

53
54
55
56
```
conda install -c pytorch torchaudio
```

57
To install the latest pip wheels, run:
jamarshon's avatar
jamarshon committed
58
59

```
60
61
pip install torchaudio -f https://download.pytorch.org/whl/torch_stable.html
```
jamarshon's avatar
jamarshon committed
62

63
64
65
(If you do not have torch already installed, this will default to installing
torch from PyPI. If you need a different torch configuration, preinstall torch
before running this command.)
jamarshon's avatar
jamarshon committed
66

67
68
69
70
71
### Nightly build

Note that nightly build is build on PyTorch's nightly build. Therefore, you need to install the latest PyTorch when you use nightly build of torchaudio.

**pip**
jamarshon's avatar
jamarshon committed
72

73
```
Mingbo Wan's avatar
Mingbo Wan committed
74
pip install numpy
Vincent QB's avatar
Vincent QB committed
75
pip install --pre torchaudio -f https://download.pytorch.org/whl/nightly/torch_nightly.html
jamarshon's avatar
jamarshon committed
76
77
```

78
**conda**
Mingbo Wan's avatar
Mingbo Wan committed
79
80
81
82
83

```
conda install -y -c pytorch-nightly torchaudio
```

jamarshon's avatar
jamarshon committed
84
85
### From Source

86
The build process builds libsox and some codecs that torchaudio need to link to. This is achieve by setting the environment variable `BUILD_SOX=1`.
moto's avatar
moto committed
87
The build process will fetch and build libmad, lame, flac, vorbis, opus, and libsox before building extension. This process requires `cmake` and `pkg-config`.
88

89
90
91
92
93
94
```bash
# Linux
BUILD_SOX=1 python setup.py install

# OSX
BUILD_SOX=1 MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py install
95
96
97
98
99
100
101
102
103

# Windows
# We need to use the MSVC x64 toolset for compilation, with Visual Studio's vcvarsall.bat or directly with vcvars64.bat.
# These batch files are under Visual Studio's installation folder, under 'VC\Auxiliary\Build\'.
# More information available at:
#   https://docs.microsoft.com/en-us/cpp/build/how-to-enable-a-64-bit-visual-cpp-toolset-on-the-command-line?view=msvc-160#use-vcvarsallbat-to-set-a-64-bit-hosted-build-architecture
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" x64 && set BUILD_SOX=0 && python setup.py install
# or
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvars64.bat" && set BUILD_SOX=0 && python setup.py install
104
105
106
```

This is known to work on linux and unix distributions such as Ubuntu and CentOS 7 and macOS.
107
If you try this on a new system and find a solution to make it work, feel free to share it by opening an issue.
108
109
110
111
112
113
114
115
116
117
118
119
120
121

#### Troubleshooting

<Details><Summary>checking build system type... ./config.guess: unable to guess system type</Summary>

Since the configuration file for codecs are old, they cannot correctly detect the new environments, such as Jetson Aarch. You need to replace the `config.guess` file in `./third_party/tmp/lame-3.99.5/config.guess` and/or `./third_party/tmp/libmad-0.15.1b/config.guess` with [the latest one](https://github.com/gcc-mirror/gcc/blob/master/config.guess).

See also: [#658](https://github.com/pytorch/audio/issues/658)

</Details>

<Details><Summary>Undefined reference to `tgetnum' when using `BUILD_SOX`</Summary>

If while building from within an anaconda environment you come across errors similar to the following:
122
123
124
125
126
127
128
129
130
131
132
133

```
../bin/ld: console.c:(.text+0xc1): undefined reference to `tgetnum'
```

Install `ncurses` from `conda-forge` before running `python setup.py install`:

```
# Install ncurses from conda-forge
conda install -c conda-forge ncurses
```

134
135
</Details>

136

Soumith Chintala's avatar
Soumith Chintala committed
137
138
139
140
141
Quick Usage
-----------

```python
import torchaudio
142
143
144

waveform, sample_rate = torchaudio.load('foo.wav')  # load tensor from file
torchaudio.save('foo_save.wav', waveform, sample_rate)  # save tensor to file
Soumith Chintala's avatar
Soumith Chintala committed
145
146
```

147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
Backend Dispatch
----------------

By default in OSX and Linux, torchaudio uses SoX as a backend to load and save files.
The backend can be changed to [SoundFile](https://pysoundfile.readthedocs.io/en/latest/)
using the following. See [SoundFile](https://pysoundfile.readthedocs.io/en/latest/)
for installation instructions.

```python
import torchaudio
torchaudio.set_audio_backend("soundfile")  # switch backend

waveform, sample_rate = torchaudio.load('foo.wav')  # load tensor from file, as usual
torchaudio.save('foo_save.wav', waveform, sample_rate)  # save tensor to file, as usual
```

Unlike SoX, SoundFile does not currently support mp3.

Soumith Chintala's avatar
Soumith Chintala committed
165
API Reference
Vincent QB's avatar
Vincent QB committed
166
-------------
SeanNaren's avatar
SeanNaren committed
167

168
API Reference is located here: http://pytorch.org/audio/
Vincent QB's avatar
Vincent QB committed
169
170
171
172

Conventions
-----------

jamarshon's avatar
jamarshon committed
173
With torchaudio being a machine learning library and built on top of PyTorch,
174
torchaudio is standardized around the following naming conventions. Tensors are
175
assumed to have "channel" as the first dimension and time as the last
jamarshon's avatar
jamarshon committed
176
177
178
dimension (when applicable). This makes it consistent with PyTorch's dimensions.
For size names, the prefix `n_` is used (e.g. "a tensor of size (`n_freq`, `n_mel`)")
whereas dimension names do not have this prefix (e.g. "a tensor of
179
dimension (channel, time)")
jamarshon's avatar
jamarshon committed
180

181
* `waveform`: a tensor of audio samples with dimensions (channel, time)
jamarshon's avatar
jamarshon committed
182
* `sample_rate`: the rate of audio dimensions (samples per second)
183
184
* `specgram`: a tensor of spectrogram with dimensions (channel, freq, time)
* `mel_specgram`: a mel spectrogram with dimensions (channel, mel, time)
jamarshon's avatar
jamarshon committed
185
186
187
188
189
190
191
* `hop_length`: the number of samples between the starts of consecutive frames
* `n_fft`: the number of Fourier bins
* `n_mel`, `n_mfcc`: the number of mel and MFCC bins
* `n_freq`: the number of bins in a linear spectrogram
* `min_freq`: the lowest frequency of the lowest band in a spectrogram
* `max_freq`: the highest frequency of the highest band in a spectrogram
* `win_length`: the length of the STFT window
192
* `window_fn`: for functions that creates windows e.g. `torch.hann_window`
jamarshon's avatar
jamarshon committed
193

Vincent QB's avatar
Vincent QB committed
194
Transforms expect and return the following dimensions.
jamarshon's avatar
jamarshon committed
195

196
197
198
199
200
201
202
203
204
205
* `Spectrogram`: (channel, time) -> (channel, freq, time)
* `AmplitudeToDB`: (channel, freq, time) -> (channel, freq, time)
* `MelScale`: (channel, freq, time) -> (channel, mel, time)
* `MelSpectrogram`: (channel, time) -> (channel, mel, time)
* `MFCC`: (channel, time) -> (channel, mfcc, time)
* `MuLawEncode`: (channel, time) -> (channel, time)
* `MuLawDecode`: (channel, time) -> (channel, time)
* `Resample`: (channel, time) -> (channel, time)
* `Fade`: (channel, time) -> (channel, time)
* `Vol`: (channel, time) -> (channel, time)
jamarshon's avatar
jamarshon committed
206

Vincent QB's avatar
Vincent QB committed
207
Complex numbers are supported via tensors of dimension (..., 2), and torchaudio provides `complex_norm` and `angle` to convert such a tensor into its magnitude and phase. Here, and in the documentation, we use an ellipsis "..." as a placeholder for the rest of the dimensions of a tensor, e.g. optional batching and channel dimensions.
208

jamarshon's avatar
jamarshon committed
209
210
211
Contributing Guidelines
-----------------------

Nicolas Hug's avatar
Nicolas Hug committed
212
Please refer to [CONTRIBUTING.md](./CONTRIBUTING.md)
Vincent QB's avatar
Vincent QB committed
213
214
215
216
217
218
219

Disclaimer on Datasets
----------------------

This is a utility library that downloads and prepares public datasets. We do not host or distribute these datasets, vouch for their quality or fairness, or claim that you have license to use the dataset. It is your responsibility to determine whether you have permission to use the dataset under the dataset's license.

If you're a dataset owner and wish to update any part of it (description, citation, etc.), or do not want your dataset to be included in this library, please get in touch through a GitHub issue. Thanks for your contribution to the ML community!