GitLab

Co-authored-by: linqj3 <linqj3@lenovo.com>

and add test on other backend.

* remove * in import of models.

* only importing WaveRNN in tests.

* Suggested changes to speed up phaser

* Checkpoint

* Checkpoint

* Checkpoint

* Checkpoint

* removing todo items
Co-authored-by: Vincent QB <vincentqb@users.noreply.github.com>

As a part of the "sox" backend sunset plan (#903), we add a "soundfile" backend that is compatible with the "sox_io" backend. No new public backend name is added. We provide a switch to change the interface/behavior of "soundfile" backend.

This commit contains;
 - The implementation of the new "soundfile" backend.
 - The flag to switch the behavior of "soundfile" backend. (`torchaudio.USE_SOUNDFILE_LEGACY_INTERFACE`)
 - Test for the new backend and switching mechanism.

The default behavior of "soundfile" backend is not changed. The users who want to opt-in the new "soundfile" interface can do so by `torchaudio.USE_SOUNDFILE_LEGACY_INTERFACE = False` before changing the backend to "soundfile".

In 0.8.0 release, the "soundfile" backend will use this interface by default, and users can still use the legacy one with `torchaudio.USE_SOUNDFILE_LEGACY_INTERFACE = True`. In 0.9.0, the legacy interface is removed and `torchaudio.USE_SOUNDFILE_LEGACY_INTERFACE` flag will be eventually removed.

* example pipeline, initial commit.

* removing notebook conversion artifacts.

* remove extra comments. lint.

* addressing some feedback.

* main function.

* defining args in function.

* refactor.

* lint.

* checkpoint.

* clean version to start with.

* adding more parameters.

* lint.

* cleaning full version.

* check for not None.

* cleaning.

* back -l 160

* black.

* fix runtime error.

* removing some print statements.

* add help to command line. add progress bar option.

* grouping librispeech-specific transform in subclass.

* typo.

* fix concatenation.

* typo.

* black. tqdm.

* missing transpose.

* renaming variables.

* sum cer and wer

* clip norm.

* second signal handler removed.

* cosmetic.

* default to no checkpoint.

* remove non_blocking.

* adadelta works better than sgd.

* anomaly detection.

* moving dataset to separate file.

* lint.

* move to separate module: languagemodel, decoder, metric.

* flush=True.

* renaming decoder.

* CTC Decoders.

* flush=True.

* pass length for viterbi decoder.

* progress bar. relative path.

* generalize transition matrix to n-gram. progress bar.

* choice of decoder.

* collate func.

* remove signal handling.

* adding distributed.

* lint.

* normalize w/r to length of dataset, and w/r to total number characters.

* relative cer/wer.

* clip grad parameter. momentum back but not yet used.

* Switch to SGD.

* choice of optimizer.

* scheduler.

* move to utils file.

* metric log, and utils file.

* rename metric_logger.

* stderr and stdout. simpler metric logger.

* replace by logging.

* adding time measurement in metric logger.

* fix duplicate name. remove tqdm. keep track of epoch instead and iteration instead.

* rename main file. and add readme.

* refactor distributed.

* swap example and output in readme.

* remove time from logger.

* check non-empty tensor input.

* typo in variable name and log update.

* typo.

* compute cer/wer in training too.

* typo.

* add back slurm signal capture to resubmit job.

* update levinstein distance.

* adding tests for levenstein distance.

* record error rate during iteration.

* metric logger using setitem.

* moving signal break to end of loop and return loss so far.

* typo.

* add citation.

* change default to best run.

* adding other experiment with decoders.

* remove other decoders than greedy.

* Revert "remove other decoders than greedy."

This reverts commit fb114372e89e317bf48d0b1f846c60bca8efe1ac.

* changing name of folfder.

* remove other decoders, and unused dataset class.

* rename functions to align with other pipeline.

* pick which parts to train with.

* adding specaugment to validation. note that caching prevents randomization from happening in validation.

* updating readme.

* typo in metric logging.

* Revert "typo in metric logging."

This reverts commit acac245eec250f61d2039a67933d3c01f1975ce9.

* Revert "Revert "typo in metric logging.""

This reverts commit 2c80d9691ed401044da734c40df3715dba92d0db.

* update metric logger.

* simplify metric logger implementation.

* use json dumps instead.

* group metric together.

* move function.

* lint.

* quick summary of files in folder.

* pass clip_grad explictly.

* typo in default dataset name.

* option to disable logger.

* ergonomics for distributed.

* reminder about signal handler.

* minor refactor of main in main.

* replace by not_main_rank.

* raising error if parameter not supported.

* move model before invoking DDP.

* changing log level. using python 2 style string for logging.

* dynamic augmentations.

* update metric log.

batch cer/wer metric. correct typo in time. adding other dimensions in metric.

* save learning rate even if function not available.

* add type option to model.

* add adamw.

* reduce lr on validation step or training step.

* specify hop-length and win-length.

* normalize option.

* rename parameter.

* add dropout and tweak to number of channels.

* copy model in pipeline folder for experimentation.

* fix scheduler stepping.

* fix input_type and num_features.

* waveform mode changes shape more.

* adding best character error rate with current implementation of model with mfcc.

* comment update.

* remove signal. remove custom wav2letter model.

* remove comment.

* simpler import with pandas.

This reverts commit 1ecbc249.

Reason: The macOS CI jobs are stuck with xdist recently.

* Warn if create_fb_matrix produces a column whose weights are all zeros

See also https://github.com/librosa/librosa/issues/478

Default to using the `cpu` directory since the base one is not reliable
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

* Add deprecation warning to sox backend

Refer to https://github.com/pytorch/audio/issues/903

* updated the build_generator call to include the models argument

* fixed RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same

Upstream is on 9.4.1 and we were experiencing issues when conda-build
was attempting to check for overlinking and failed out due to some files
not existing in xcode 9.0
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

When running smoke test, proper way to install conda package is to
install from workspace, instead of installing from archive.
Otherwise the dependencies are not properly installed

https://github.com/conda/conda/issues/1884

For this, we pass the whole `conda-bld` directory from
conda package build job to upload job then smoke test job.

`torchaudio` does not have a mandatory third-party python package dependency,
so this was not an issue.

See also https://github.com/pytorch/text/pull/803

* Fix incorrect extension parsing in sox_io_backend.save
* Add tests for compression=None

* Tweak docstring, audio_ext, load method signature and constructor of VCTK_092

* Add test for VCTK_092 dataset.

* Added version 0.92 of VCTK dataset
Signed-off-by: Abhishek Dubey <abhi.dubey011999@gmail.com>

librosa 0.8.0 has been released which include fix for
https://github.com/librosa/librosa/issues/1160 at
https://github.com/librosa/librosa/pull/1164

* Add spectrogram normalization option
Co-authored-by: Ji Chen <jimchen90@devfair0160.h2.fair>

This CC adds CI test to check the backward compatibility of Torchscript functions/object dumps.

The job first dumps Torchscript objects into file from torchaudio 0.6.0 release environment (x Python 3.6, 3.7, 3,8), then load & run the function in master build (in Python 3.6, 3.7, 3.8).

If there is a BC-breaking change in master build, (registration schema change), then the test should fail.

At this moment, `info` function does not work due to suspected bug in torch side, so the test is disabled for `info` function. See https://github.com/pytorch/pytorch/issues/42258 for the detail. Once https://github.com/pytorch/pytorch/issues/42258 is resolved we can enable it.

We have been running unit test with editable installation. (i.e. `python setup.py develop`), with which we missed issues like #842. 

This CC makes installation in CI non-editable, and change test directory structure so that the source code will not shadow the installed version of `torchaudio`. With simple `pytest test`, `pytest` modifies `sys.path` and prepend checked out repository, which shadows the installed version.

To remedy this, the whole test suites has been moved from `./test` to `./test/torchaudio_unittest`. This adds nice module structure to our test code and we can do absolute import in each test module, which makes it possible again to run test with `python -m unittest torchaudio_unittest/XXX.py`

This change does not affect the regular development process (`python setup.py develop` && `pytest test`)