GitLab

Summary:
Add a new design of MVDR module.
The RTFMVDR module supports the method based on the relative transfer function (RTF) and power spectral density (PSD) matrix of noise.
The input arguments are:
- multi-channel spectrum.
- RTF vector of the target speech
- PSD matrix of noise.
- reference channel in the microphone array.
- diagonal_loading option to enable or disable diagonal loading in matrix inverse computation.
- diag_eps for computing the inverse of the matrix.
- eps for computing the beamforming weight.
The output of the module is the single-channel complex-valued spectrum for the enhanced speech.

Pull Request resolved: https://github.com/pytorch/audio/pull/2368

Reviewed By: carolineechen

Differential Revision: D36214940

Pulled By: nateanl

fbshipit-source-id: 5f29f778663c96591e1b520b15f7876d07116937

Summary:
Add a new design of MVDR module.
The `SoudenMVDR` module supports the method proposed by [Souden et, al.](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.725.673&rep=rep1&type=pdf).
The input arguments are:
- multi-channel spectrum.
- PSD matrix of target speech.
- PSD matrix of noise.
- reference channel in the microphone array.
- diagonal_loading option to enable or disable diagonal loading in matrix inverse computation.
- diag_eps for computing the inverse of the matrix.
- eps for computing the beamforming weight.

The output of the module is the single-channel complex-valued spectrum for the enhanced speech.

Pull Request resolved: https://github.com/pytorch/audio/pull/2367

Reviewed By: hwangjeff

Differential Revision: D36198015

Pulled By: nateanl

fbshipit-source-id: 4027f4752a84aaef730ef3ea8c625e801cc35527

Summary:
- Add linear_fbank method
- Add LFCC in transforms

* Use `bibtex` for paper citations.
  * add `override.css` for fixing back reference.
  * wav2vec2
  * wav2letter
  * convtasnet
  * deepspeech
  * rnnt-loss
  * griffinlim
* Fix broken references in `filtering`.
* Fix note in soundfile backends.
* Tweak wav2vec2 example.
* Removes unused `pytorch_theme.css`

* initial test, stub function, transform and docstring

* add draft working implementation, update docstrings

* merge VadSate into Vad calss, move Channel into Vad class

* remove functional stub for vad

* add wav file for test

* refactor _measure() to improve performance

* rename argument

* replace copy_ with assignment

* refactor init, update documentation, update test for readability

* clean up default values

* move code from transforms.py to funtional.py and integrate state into a function

* remove Channel state class

* fix calcuation of a flush point

* make multiple channels work

* clean up multi-channel, update test

* rename variables and re-org arguments for _measure

* fix linting errors

* add torchscript consistency test and fix errors

* support and test batch consistency, fix normalization

* update documentation, switch torchscript consistancy test to use transform to improve coverage

* fix linting errors

* remove un-used imports

* address PR comments

* add doc references into rst

* add cmvn

* Update transforms.rst

add cmvn

* Correct the format

* Correct the format

* Correct the format

* add test unit and cmvn change to cmn

* fix bug
Co-authored-by: Vincent QB <vincentqb@users.noreply.github.com>

* Add Vol with gain_type amplitude

* add gain in db and add tests

* add gain_type "power" and tests

* add functional DB_to_amplitude

* simplify

* remove functional

* improve docstring

* add to documentation

* add basics for Fade

* add fade possibilities: at start, end or both

* add different types of fade

* add docstrings, add overriding possibility

* remove unnecessary logic

* correct typing

* agnostic to batch size or n_channels

* add batch test to Fade

* add transform to options

* add test_script_module

* add coherency with test batch

* remove extra step for waveform_length

* update docstring

* add test to compare fade with sox

* change name of fade_shape

* update test fade vs sox with new nomenclature for fade_shape

* add Documentation
Co-authored-by: Vincent QB <vincentqb@users.noreply.github.com>

* Inverse Mel Scale Implementation

* Inverse Mel Scale Docs

* Better working version.

* GPU fix

* These shouldn't go on git..

* Even better one, but does not support JITability.

* Remove JITability test

* Flake8

* n_stft is a must

* minor clean up of initialization

* Add librosa consistency test

This PR follows up #366 and adds test for `InverseMelScale` (and `MelScale`) for librosa compatibility.

For `MelScale` compatibility test;
1. Generate spectrogram
2. Feed the spectrogram to `torchaudio.transforms.MelScale` instance
3. Feed the spectrogram to `librosa.feature.melspectrogram` function.
4. Compare the result from 2 and 3 elementwise.
Element-wise numerical comparison is possible because under the hood their implementations use the same algorith.

For `InverseMelScale` compatibility test, it is more elaborated than that.
1. Generate the original spectrogram
2. Convert the original spectrogram to Mel scale using `torchaudio.transforms.MelScale` instance
3. Reconstruct spectrogram using torchaudio implementation
3.1. Feed the Mel spectrogram to `torchaudio.transforms.InverseMelScale` instance and get reconstructed spectrogram.
3.2. Compute the sum of element-wise P1 distance of the original spectrogram and that from 3.1.
4. Reconstruct spectrogram using librosa
4.1. Feed the Mel spectrogram to `librosa.feature.inverse.mel_to_stft` function and get reconstructed spectrogram.
4.2. Compute the sum of element-wise P1 distance of the original spectrogram and that from 4.1. (this is the reference.)
5. Check that resulting P1 distance are in a roughly same value range.

Element-wise numerical comparison is not possible due to the difference algorithms used to compute the inverse. The reconstructed spectrograms can have some values vary in magnitude.
Therefore the strategy here is to check that P1 distance (reconstruction loss) is not that different from the value obtained using `librosa`. For this purpose, threshold was empirically chosen

```
print('p1 dist (orig <-> ta):', torch.dist(spec_orig, spec_ta, p=1))
print('p1 dist (orig <-> lr):', torch.dist(spec_orig, spec_lr, p=1))
>>> p1 dist (orig <-> ta): tensor(1482.1917)
>>> p1 dist (orig <-> lr): tensor(1420.7103)
```

This value can vary based on the length and the kind of the signal being processed, so it was handpicked.

* Address review feedbacks

* Support arbitrary batch dimensions.

* Add batch test

* Use view for batch

* fix sgd

* Use negative indices and update docstring

* Update threshold
Co-authored-by: Charles J.Y. Yoon <jaeyeun97@gmail.com>

* Griffin-Lim Transformation Implementation

* Griffin-Lim Docs

* Remove f-string from backwards compatibility

* iSTFT is now jit-able.

* Comment changes

* Functional Implementation & now jitable

* flake8

* Doc & GPU Fix

* Librosa comparison test

* test directly griffinlim's output. tighter atol.

* matching signature to docstring.
Co-authored-by: Vincent QB <vincentqb@users.noreply.github.com>

* since we no longer use decoration, this fixes #165.

* remove import of _docs.

* sync docs with functionals.

* Adding transforms to documentations. Moving augmentations in transforms.