GitLab

Currently all the tests in `sox_io_backend` and `sox_effects` (for new SoX effects implementation) requires additional `sox`, and this prevents running test in environment where `sox` command is not available even though `torchaudio` extension is available (such as fb internal). This PR adds smoke tests for these modules, which just runs functions to see if they do not crash.

* Addressed review issues in PR #668

* Changed GTZAN so that it only traverses filenames belonging to the dataset

Now, instead of walking the whole directory and subdirectories of the dataset
GTZAN only looks for files under a `genre`/`genre`.`5 digit number`.wav format, where `genre` is an allowed GTZAN genre label.
This allows moving or removing files from the dataset (e.g. for fixing duplication or mislabeling issues).

* Get rid of sine wave files and whitenoise files
* Refactor integer encoding
* Relax rtol from 1e-8 to 1e-7 for compliance kaldi
* relax waveform multi channel resample atol to 1e-7 from 1e-8
* relax tolerance for length consistency for speed effect
Co-authored-by: moto <855818+mthrok@users.noreply.github.com>

* Add sox_utils module

* Make init/shutdown thread safe

* Add sox effects implementation

* Add test for sox effects

* Update docstrings and add examples

* converted CommonVoice tartar mp3 to wav using rate 8000 Hz

* Remove Unused dtmf_30s_stereo.mp3

* Add cmu_arctic dataset

* add dataset name

* update audio test file with whitenoise.wav file

* add test text file

* update text method and file name

* update comment

* change datasets order in doc

* add line length
Co-authored-by: Ji Chen <jimchen90@devfair0160.h2.fair>

* kaldi compliance files cleanup for spec, fbank, mfcc

* kaldi compliance tests removal for spec, fbank, mfcc
Signed-off-by: Bhargav Kathivarapu <bhargavkathivarapu31@gmail.com>

* Migrate spectrogram

* Update spectrogram in kaldi.py to support device and dtype

* Remove failing tests
Signed-off-by: Bhargav Kathivarapu <bhargavkathivarapu31@gmail.com>

* Fix device/dtype compatibility of Kaldi mfcc

* Migrate Kaldi mfcc test

* Remove failing tests
Signed-off-by: Bhargav Kathivarapu <bhargavkathivarapu31@gmail.com>

* Migrate fbank tests

* Update CCI job environment

* Remove invalid test cases
Signed-off-by: Bhargav Kathivarapu <bhargavkathivarapu31@gmail.com>

* Added the popular GTZAN dataset:

* Added the GTZAN class in torchaudio.datasets using the same format as the rest of the datasets.
* Added the appropriate test function in test_datasets.py.
* Added the GTZAN class in the datasets.rst documentation file.

* Addressed review issues in PR #668

* Added dummy noise .wav in `test/assets/`
* Removed transforms of input and output from the dataset
  `__init__` function, as well as the corresponding methods.
* Replaced rendundant `filtered` and `subset` methods from
  class initialization and also changed the corresponding
  assertion message.

* Fixed E303: too many blank lines error

* Added GTZAN to __init__.__all__

* Fixed incorrectly not importing GTZAN

* removed duplicate warning

* lint
Co-authored-by: Vincent QB <vincentqb@users.noreply.github.com>

* initial test, stub function, transform and docstring

* add draft working implementation, update docstrings

* merge VadSate into Vad calss, move Channel into Vad class

* remove functional stub for vad

* add wav file for test

* refactor _measure() to improve performance

* rename argument

* replace copy_ with assignment

* refactor init, update documentation, update test for readability

* clean up default values

* move code from transforms.py to funtional.py and integrate state into a function

* remove Channel state class

* fix calcuation of a flush point

* make multiple channels work

* clean up multi-channel, update test

* rename variables and re-org arguments for _measure

* fix linting errors

* add torchscript consistency test and fix errors

* support and test batch consistency, fix normalization

* update documentation, switch torchscript consistancy test to use transform to improve coverage

* fix linting errors

* remove un-used imports

* address PR comments

* add doc references into rst

* fix download

* fix reading tsv archive

* add new languages

* maintain same structure as other datasets

* update CommonVoice Tests

* fix

* change directory name

* remove extra line

* testing with sox only when sox is available.

* use wav instead of mp3 for testing functions.

* typo.

* guard against not sox.

* backends definition.

* grouping backend functions into a separate file.

* remove duplicated code.

* requires sox.

* replace by wav, requires sox.

* require with scope.

* undo alignment.

* requires sox for these two, because of mp3.

* no longer need first mp3.

* cleaning.

* new wav version of file.

* flake8.

* remove unnecessary load.

* flake8.

* lint.

* lint.

* revert formatting of file.

* merging into common_utils.

* docstring.

* rename to common_utils.

* add speechcommand dataset and test

* prepend the full path to each result

* add missing param on docstring in walk_files

* add file to run tests on SpeechCommand Dataset

* reduce logic

* update test on SpeechCommands

* correct the indentation on docstring walk_files

* flake8 compliance

* change tuple type returned. move path split logic in load item.

* typo in name.

* redundant file path.

* filter background noise.
Co-authored-by: Vincent QB <vincentqb@users.noreply.github.com>

* LJ Speech dataset

* refactoring

as per @vincentqb's suggestions

* move sox inside function calls.

* add backend switch mechanism.

* import sox at runtime, not import.

* add backend list.

* backend tests.

* creating hidden modules for backend.

* naming backend same as file: soundfile.

* remove docstring in backend file.

* test soundfile info.

* soundfile doesn't support int64.

* adding test for wav file.

* error with incorrect parameter instead of silent ignore.

* adding test across backend. using float32 as done in sox.

* backend guard decorator.

* BackgroundGenerator
* renaming disk cache.

Fix test for datasets.

* new dataset format.
* add basic test.
* files for testing.
* serialization using torch.
* add diskcache.
* adding deprecation warnings.
* removing legacy.
* warning about transforms.
* detecting file format using reader.

* Add basic low pass filtering
* Add highpass filtering
* More tests of IIR vs FIR
* Implement convolve function, add tests
* Move lfilter and convolve into functional, more tests
* added additional documentation for convolve and lfilter, renamed functional_filtering to functional_sox_convenience
* Follow naming convention for sample rate in functional
* fix failing vctk manifest test to account for adding more test audios into assets
* Adding documentation for lfilter, biquad, highpass_biquad, lowpass_biquad
* added matrix based implementation of lfilter
* adding python lfilter implementation
* factor out biquad, lowpass, highpass to sox compatibility

Add Kaldi IO as a dependency + put  a wrapper to convert to Tensor + add test to check correct type (#111)

This reverts commit cfe3f296.

* Revert "Revert "Rewrote C code with C++ extensions" (#39)"

This reverts commit 7236e3f4

.

* Fix import error of C extension
Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Revert "Nits"

This reverts commit 2db284c4.

* Revert "Conform to sox library a bit better"

This reverts commit c340a8d1.

* Revert "Added test to make sure loading and saving gives the same file"

This reverts commit cba11009.

* Revert "Rewrote C code with C++ extensions"

This reverts commit 8a41ecdc.

vctk dataset

add label processing

added tests

chunk dataset

simple caching scheme

caching fixed

vctk downsample

yesno dataset