GitLab

We have been running unit test with editable installation. (i.e. `python setup.py develop`), with which we missed issues like #842. 

This CC makes installation in CI non-editable, and change test directory structure so that the source code will not shadow the installed version of `torchaudio`. With simple `pytest test`, `pytest` modifies `sys.path` and prepend checked out repository, which shadows the installed version.

To remedy this, the whole test suites has been moved from `./test` to `./test/torchaudio_unittest`. This adds nice module structure to our test code and we can do absolute import in each test module, which makes it possible again to run test with `python -m unittest torchaudio_unittest/XXX.py`

This change does not affect the regular development process (`python setup.py develop` && `pytest test`)

Currently `setuptools` assumes that torhcaudio is zip safe and performs
egg (zip) installation when `python setup.py install` (or `pip install .`).

When `torchaudio` loads extension module written with Torchscript, 
the corresponding loading function requires the `so` file to be present as 
an actual file, and this does not work well with egg installation.

This CC fixes this by setting `zip_safe=False` in setup.py and disable egg
installation, so that `python setup.py install` installs uncompressed files
with regular directory structure.

Currently our test suites automatically/silently skip tests on C++ extension
if it is not available. This is nice in local env, but in CI these tests should be
enforced and reported as failure if C++ extension is not available.

This PR adds switch for making tests fail if C++ extension is not available,
and make CI for *nix fail if that's the case.

setuptools warns that it should be imported before Distutils

```
site-packages/setuptools/distutils_patch.py:25: UserWarning: Distutils was imported before Setuptools. This usage is discouraged and may exhibit undesirable behaviors or errors. Please use Setuptools' objects directly or at least import Setuptools first.
```

* Update links
* Add how to run tests
* Update asset info

* `istft` has been migrated to `pytorch`, and `torchaudio.functional.istft` has been deprecated in 0.6.0 release. This PR removes it
Co-authored-by: Jeremy Chen <jeremyyy@fb.com>

Co-authored-by: Ji Chen <jimchen90@devfair0160.h2.fair>

Currently macOS CI job takes twice the time of linux unit tests.
This PR uses `pytest-xdist` to run unit tests in parallel on macOS CI job. 

Interestingly applying the same technique to linux unit tests make the test execution longer.
So the change is only applied to macOS.

macOS 27 mins -> 17 mins 😸
linux 14 mins -> 1 hours 😾

Now that most of our test cases are backend aware/agnostic,
the use of BACKENDS and BACKENDS_MP3 is anti-pattern.

This change moves these definitions to the legacy test module `test_io`.

If a new backend is added in future, that backend should have separate test suite,
(like the ones in `test/sox_io_backend`) so instead of dynamically listing available
backend in `test_io`, the changed code only checks "sox" backend and "soundfile" backend.

Co-authored-by: Ji Chen <jimchen90@devfair0160.h2.fair>

* Remove underscore of model name
Co-authored-by: Ji Chen <jimchen90@devfair0160.h2.fair>

* Add test for CommonVoice dataset

* Migrate the existing tests for `bg_iterator` and `diskcache_iterator` to `test/datasets/utils_test.py`
Co-authored-by: Leon Gao <legao@linkedin.com>

Co-authored-by: lawrencechen <lawrencechen@devvm3189.vll0.facebook.com>

Co-authored-by: Edward Lu <edlu@fb.com>

* Update sample rate of libritts test
Co-authored-by: Ji Chen <jimchen90@devfair0160.h2.fair>

Buck fails to run tests if they contain "-" symbol.

* Replace save/load function with sox effects chain

Part of #764

* Separate sox list format function for read and write

* Guard MP3 smoke test

buck gets confused with utility module name with `test_` prefix.

1. Move misplaced sox compatibility test (T,Fade, T.Vol, T.Vad) to test/test_sox_compatibility.py
2. Move test_sox_effects to test/sox_effect/ where all the other functionalities from torchaudio.sox_effects are tested

Currently all the tests in `sox_io_backend` and `sox_effects` (for new SoX effects implementation) requires additional `sox`, and this prevents running test in environment where `sox` command is not available even though `torchaudio` extension is available (such as fb internal). This PR adds smoke tests for these modules, which just runs functions to see if they do not crash.

Co-authored-by: Jeff Zhang <jeffzhang@fb.com>

* Add WaveRNN example

This is the pipeline example based on [WaveRNN model](https://github.com/pytorch/audio/pull/735) in torchaudio. The design of this pipeline is inspired by [#632](https://github.com/pytorch/audio/pull/632). It offers a standardized implementation of WaveRNN vocoder in torchaudio.

* Add utils and readme

The metric logger is added based on the Wav2letter pipeline [#632](https://github.com/pytorch/audio/pull/632). It offers the way to parse the standard output as described in readme.

* Add channel dimension

The channel dimension of waveform in datasets is added to match the input dimensions of WaveRNN model because the channel dimensions of waveform and spectrogram are added in [this part] (https://github.com/pytorch/audio/blob/master/torchaudio/models/_wavernn.py#L281) of WaveRNN model.

* Update date split and transform

The design of dataset structure is discussed in [this comment](https://github.com/pytorch/audio/pull/749#discussion_r454627027

). Now the dataset file has a clearer workflow after using the random-split function instead of walking through all the files. All transform functions are put together inside the transforms block.
Co-authored-by: Ji Chen <jimchen90@devfair0160.h2.fair>

- Addresses #549 #638 #786 
- Add `torchaudio` top level module doc
- Separate `torchaudio` top level module doc from `index.html`
- Add `backend` module doc.
- Remove `-> None` from function signature as it adds noise to documentation
- Changed function argument name of `torchaudio.backend.sox_io_backend.save` from `tensor` to `src`, so that it matches with the reset of backends.
- Tweak bunch of docstrings

Fix output type of upsampling
Co-authored-by: Ji Chen <jimchen90@devfair0160.h2.fair>

* Update default form in docstring
Co-authored-by: Ji Chen <jimchen90@devfair0160.h2.fair>

* Add libritts

Add LibriTTS dataset draft

* Add libritts

Use two separate ids for utterance_id.

* Update output form

Use full_id as utterance_id.

* Update format

Add space and test black format

* Update test method

* Add audio and text test

Generate audio and test files on-the-fly in test 

* Update format

* Fix test error and remove assets libritts

The test error is fixed by sorting the file in 4th element instead of 2nd element in samples. Since the files are generated on-the-fly, so the the libritts files in assets are removed.

* Add seed in `get_whitenoise` function

* Change utterance to text

Change `_utterance` to `_text`.
Co-authored-by: Ji Chen <jimchen90@devfair0160.h2.fair>

* Change the name of  n_output and n_hidden

* Replace the mode by n_classes and sample_rate

* Change the definition of n_output and n_hidden
Co-authored-by: Ji Chen <jimchen90@devfair0160.h2.fair>

* Addressed review issues in PR #668

* Changed GTZAN so that it only traverses filenames belonging to the dataset

Now, instead of walking the whole directory and subdirectories of the dataset
GTZAN only looks for files under a `genre`/`genre`.`5 digit number`.wav format, where `genre` is an allowed GTZAN genre label.
This allows moving or removing files from the dataset (e.g. for fixing duplication or mislabeling issues).