GitLab

Summary:
Part of https://github.com/pytorch/audio/issues/2072 -- splitting up PR for easier review

This PR adds Python decoder API and basic README

Pull Request resolved: https://github.com/pytorch/audio/pull/2089

Reviewed By: mthrok

Differential Revision: D33299818

Pulled By: carolineechen

fbshipit-source-id: 778ec3692331e95258d3734f0d4ab60b6618ddbc

Summary:
Pull Request resolved: https://github.com/pytorch/audio/pull/2096

run: `arc lint --apply-patches --paths-cmd 'hg files -I "./**/*.py"'`

Reviewed By: mthrok

Differential Revision: D33297351

fbshipit-source-id: 7bf5956edf0717c5ca90219f72414ff4eeaf5aa8

Summary:
Adds implementation of Conformer module.

Adapted from sravyapopuri388's implementation for fairseq at https://github.com/fairinternal/fairseq-py/pull/2770.

Pull Request resolved: https://github.com/pytorch/audio/pull/2068

Reviewed By: mthrok

Differential Revision: D33236957

Pulled By: hwangjeff

fbshipit-source-id: 382d99394996ff5249522b5899e1a4b4a95de9e6

Summary:
Adds beam search decoder for RNN-T implementation ``torchaudio.prototype.RNNT`` that is TorchScript-able and supports both streaming and non-streaming inference.

Pull Request resolved: https://github.com/pytorch/audio/pull/2028

Reviewed By: mthrok

Differential Revision: D32627919

Pulled By: hwangjeff

fbshipit-source-id: aab99e346d6514a3207a9fb69d4b42978b4cdbbd

Summary:
- Remove unnecessary content list
- Remove legacy description

Pull Request resolved: https://github.com/pytorch/audio/pull/2029

Reviewed By: carolineechen

Differential Revision: D32629917

Pulled By: mthrok

fbshipit-source-id: bc9a9366c681bcf8b74907c2a6459c73fb6a7424

Summary:
With the introduction of tutorials, the turn around time for doc build
has become longer. By default, the tutorial is not built but SPHINXOPT=-W
treats it as error.

This commit disable the option for the local build while keeping it
for the CI.

Pull Request resolved: https://github.com/pytorch/audio/pull/2013

Reviewed By: carolineechen

Differential Revision: D32538952

Pulled By: mthrok

fbshipit-source-id: eae4ffd87100dff466f91abfe26a82aa702d605a

Summary:
Adds streaming-capable recurrent neural network transducer (RNN-T) model that uses Emformer for its transcription network. Includes two factory functions — one that allows for building a custom model, and one that builds a preconfigured base model.

Pull Request resolved: https://github.com/pytorch/audio/pull/2003

Reviewed By: nateanl

Differential Revision: D32440879

Pulled By: hwangjeff

fbshipit-source-id: 601cb1de368427f25e3b7d120e185960595d2360

- Add link to index page on left
- Package Reference -> API Reference
- Update description.

* Refactor tutorial organization

* Merge tutorial subdirectoris under to examples/gallery/tutorials
* Do not use index.rst generated by Sphinx-gallery
* Instead use flat structure so that all the tutorials are listed in left menu
* Use `_assets` dir for artifacts of tutorials

The hack introduced in #1969 could break if upstream theme changes its HTML or Javascript.
To prevent our documentation from randomly break, this commit fixes the commit of the them.

This commit adds colab/download/source link to tutorials, like in `pytorch/tutorials` repo.

Since the upstream `pytorch-sphinx-theme` does not provide the interface for this,
a hack to overwrite the URL is added.

This hack might stop working if there is some update in `pytorch-sphinx-theme`.

With the introduction of TTS tutorial (#1973), it takes more than couple of minutes
to build documentation. This commit makes the doc build process defaults to
not build tutorials.

To build tutorials one can use environment variable `BUILD_GALLERY=1`,
and set `GALLERY_PATTERN=...` to filter the tutorials to build.

This `GALLERY_PATTERN` is same approach as in `tutorials` repo.

https://github.com/pytorch/tutorials/blob/cbf2238df0e78d84c15bd94288966d2f4b2e83ae/conf.py#L75-L83

Also this commit dynamically parse the subdirectory of `examples/gallery` so that when a new category of examples are added, it will automatically parsed.

* Add wav2vec2 ASR German pretrained model from voxpopuli

1. Override the return type so that Sphinx shows the exported symbols.
   (output model types and input torch.nn.Module)
2. Tweak docs for Tacotron2TTSBundle interfaces
3. Fix for HUBERT_ASR_XLARGE

Resolve the following warnings when `make clean html`.

```
parsing bibtex file /torchaudio/docs/source/refs.bib... parsed 26 entries
loading intersphinx inventory from https://docs.python.org/objects.inv...
loading intersphinx inventory from https://docs.scipy.org/doc/numpy/objects.inv...
loading intersphinx inventory from https://pytorch.org/docs/stable/objects.inv...
intersphinx inventory has moved: https://docs.python.org/objects.inv -> https://docs.python.org/3/objects.inv
intersphinx inventory has moved: https://docs.scipy.org/doc/numpy/objects.inv -> https://numpy.org/doc/stable/objects.inv
```

Future work items:
- length computation of GriffinLim
- better way to make InverseMelScale work in inference_mode

- Move wav2vec2 pretrained weights to `torchaudio.pipelines` namespace to align with #1872.
- Split `Wav2Vec2PretrainedModelBundle` into `Wav2Vec2Bundle` (for pre-training model) and  `Wav2Vec2ASRBundle` (for models fine-tuned for ASR).
- Update base URL

This commit merges wav2vec2/hubert factory functions for pre-training and fine-tuning. In #1829, we added parameters to customize the models that are not part of architecture, and `aux_num_out` falls into this category, so it is no longer necessary to have separate functions. This concludes the wav2vec2/HuBERT API update in release 0.10.

The summary of BC-breaking changes on wav2vec2 APIs between 0.9 and 0.10 (when this commit is incorporated)
1. `Wav2Vec2Model.extract_features`
In 0.9, it was returning the output from `FeatureExtractor` module. In 0.10, it returns the list of outputs from the intermediate layers of `TransformerEncoder` block.
2. `wav2vec2_base(num_out: int)` -> `wav2vec2_base(<dropout_params:float>, aux_num_out: Optional[int]=None)`
    - `num_out` was renamed to `aux_num_out` and optional. If it is omitted, the resulting model does not have the linear layer for fine-tuning.
    - Added dropout parameters.