GitLab

Summary:
ctc decoder has been moved to beta, remove prototype message from tutorial

(this is done on the release branch in https://github.com/pytorch/audio/issues/2457)

Pull Request resolved: https://github.com/pytorch/audio/pull/2459

Reviewed By: hwangjeff

Differential Revision: D36978417

Pulled By: carolineechen

fbshipit-source-id: e580c1e8475a1a0aa924d44deea3852adc332a86

Summary:
The PR contains the CTC fine-tuning recipe of HuBERT Base model.
The files include:
- lightning module
- training script
- README and the result table
- evaluation scripts

Pull Request resolved: https://github.com/pytorch/audio/pull/2352

Reviewed By: hwangjeff

Differential Revision: D36915712

Pulled By: nateanl

fbshipit-source-id: 0249635ad5e81a8aa2d228c1d5fe84d78b62a15b

Summary:
- Adopt `torchaudio.utils.download_asset` to simplify asset management.
- Break down the first section about helper functions.
- Use tempfile so that executing tutorial won't leave any artifacts on local file system.

Example: https://output.circle-artifacts.com/output/job/b11a0087-8bf9-4999-a74f-b53798eaa77f/artifacts/0/docs/tutorials/audio_io_tutorial.html

Pull Request resolved: https://github.com/pytorch/audio/pull/2385

Reviewed By: hwangjeff

Differential Revision: D36404399

Pulled By: mthrok

fbshipit-source-id: 106af34e8ddd22a061aa12767b444b32aef07bad

Summary:
At the time of release, we need to download doc built by CI.
CircleCI does not have feature to download multiple files.

This commit add the archive of built documentations as
CI artifact so that the whole documentation can be downloaded
at once.

Resolves https://github.com/pytorch/audio/issues/2340

Pull Request resolved: https://github.com/pytorch/audio/pull/2448

Reviewed By: hwangjeff

Differential Revision: D36942077

Pulled By: mthrok

fbshipit-source-id: 61dde0d71841434a3d0624404d99911aa6956f88

Summary:
Import StreamReader from the new location

Pull Request resolved: https://github.com/pytorch/audio/pull/2455

Reviewed By: nateanl

Differential Revision: D36959668

Pulled By: mthrok

fbshipit-source-id: c2b8c9f9dff1ec306ea39c495294faa9208b3c4e

Summary:
Address https://github.com/pytorch/audio/issues/2445

Pull Request resolved: https://github.com/pytorch/audio/pull/2450

Reviewed By: carolineechen

Differential Revision: D36945877

Pulled By: mthrok

fbshipit-source-id: c7f9ba8093c8dc03b27582b9c608b023c7700332

Summary:
With the default log-level, completely sane operation like converting
YUV to RGB issues bunch of warnings like

`[swscaler @ 0x128aa8000] No accelerated colorspace conversion found from yuv420p to rgb24.`

This commit sets the log level to FATAL.

Pull Request resolved: https://github.com/pytorch/audio/pull/2447

Reviewed By: hwangjeff

Differential Revision: D36938728

Pulled By: mthrok

fbshipit-source-id: 39c2e6a4307f1eac577fd606e17ab0f298079b54

Summary:
Pull Request resolved: https://github.com/pytorch/audio/pull/2437

Refactors LibriSpeech Lightning datamodule to accommodate different dataset implementations.

Reviewed By: carolineechen, nateanl

Differential Revision: D36731577

fbshipit-source-id: 4ba91044311fa3f99a928aef6ef411316955f6b5

Summary:
Undesired logs are one of the loudest UX complains we get.
Yet, loading media files involves uncertainty which is
difficult to debug without debug log.

This commit introduces utility functions to configure logging level
so that we can ask users to enable it when they encounter an issue,
while defaulting to non-verbose option.

Pull Request resolved: https://github.com/pytorch/audio/pull/2439

Reviewed By: hwangjeff, xiaohui-zhang

Differential Revision: D36903763

Pulled By: mthrok

fbshipit-source-id: f4ddd9915b13197c2a2eb97e965005b8b5b8d987

Summary: Pull Request resolved: https://github.com/pytorch/audio/pull/2443

Reviewed By: nateanl

Differential Revision: D36909822

Pulled By: carolineechen

fbshipit-source-id: ef3ab2345e7a4666cf29dd02c83d03504e8aa62c

Summary:
- Adopt `torchaudio.utils.download_asset` to simplify asset management.
- Break down the first section about helper functions.
- Reduce the number of helper functions

https://output.circle-artifacts.com/output/job/d7dd1b93-6dfe-46da-a080-109bfdc63881/artifacts/0/docs/tutorials/audio_data_augmentation_tutorial.html

Pull Request resolved: https://github.com/pytorch/audio/pull/2388

Reviewed By: carolineechen

Differential Revision: D36404405

Pulled By: mthrok

fbshipit-source-id: f460ed810519797fce6e2fa7baaee110bddd1d06

Summary:
- Replace mis-use of plot_specgram with plot_sweep, and remove plot_specgram
- Move `benchmark_resample` to later section

https://output.circle-artifacts.com/output/job/9f7af187-777d-4d75-840f-2630a36295b7/artifacts/0/docs/tutorials/audio_resampling_tutorial.html

Pull Request resolved: https://github.com/pytorch/audio/pull/2386

Reviewed By: carolineechen

Differential Revision: D36404403

Pulled By: mthrok

fbshipit-source-id: f9df8453e3f531bdc4549b0134e5dbba90653bf7

Summary:
- Adopt torchaudio.utils.download_asset to simplify asset management.
- Break down the first section about helper functions.
- Reduce the number of helper functions

Pull Request resolved: https://github.com/pytorch/audio/pull/2391

Reviewed By: carolineechen, nateanl

Differential Revision: D36885626

Pulled By: mthrok

fbshipit-source-id: 1306f22ab70ab1e7f74ed7e43bf43150015448b6

Summary:
For test files where applicable, removed manual seeds where applicable. Refactoring https://github.com/pytorch/audio/issues/2267

Pull Request resolved: https://github.com/pytorch/audio/pull/2436

Reviewed By: carolineechen

Differential Revision: D36896854

Pulled By: skim0514

fbshipit-source-id: 7b4dd8a8dbfbef271f5cc56564dc83a760407e6c

Summary:
Refactor M1 logic
These improvement introduced in following PR: https://github.com/pytorch/vision/pull/6117

Pull Request resolved: https://github.com/pytorch/audio/pull/2438

Reviewed By: nateanl

Differential Revision: D36896028

Pulled By: atalman

fbshipit-source-id: 2ce360bfa78b2a7c77d5d4db800d487d171831a9

Summary:
Retrieve version from version.txt
These improvement introduced in following PR: https://github.com/pytorch/vision/pull/6117
In addition to this we add version.txt file to help us manage torchaudio version

Pull Request resolved: https://github.com/pytorch/audio/pull/2434

Reviewed By: mthrok

Differential Revision: D36867886

Pulled By: atalman

fbshipit-source-id: 14b6d653e46489d8db1c5ae2016a8202c632861e

Summary:
update QUESST14 getitem to include docstrings and additionally return sample rate

Pull Request resolved: https://github.com/pytorch/audio/pull/2435

Reviewed By: nateanl

Differential Revision: D36864254

Pulled By: carolineechen

fbshipit-source-id: 9e68bbc5de27ad2f32f6b298414103c4f6784801

Summary:
Remove the code related to libmad, which had been disabled in https://github.com/pytorch/audio/issues/2354

In https://github.com/pytorch/audio/issues/2419, we mp3 decoding to ffmpeg. But CI tests were still using libmad.
This commit completely removes libmad from torchaudio.

This is BC-breaking change as `apply_sox_effects_file` function cannot handle MP3, and it cannot fallback to ffmpeg.
The workaround for this is to use `torchaudio.load` then `apply_sox_effects_tensor`.

Pull Request resolved: https://github.com/pytorch/audio/pull/2428

Reviewed By: carolineechen

Differential Revision: D36851805

Pulled By: mthrok

fbshipit-source-id: f98795c59a1ac61cef511f2bbeac37f7c3c69d55

Summary:
- Use `download_asset` to download audios.
- Replace `MVDR` module with new-added `SoudenMVDR` and `RTFMVDR` modules.
- Benchmark performances of `F.rtf_evd` and `F.rtf_power` for RTF computation.
- Visualize the spectrograms and masks.

Pull Request resolved: https://github.com/pytorch/audio/pull/2398

Reviewed By: carolineechen

Differential Revision: D36549402

Pulled By: nateanl

fbshipit-source-id: dfd6754e6c33246e6991ccc51c4603b12502a1b5

Summary:
This commit add fallback mechanism to `info` and `load` functions of sox_io backend.
If torchaudio is compiled to use FFmpeg, and runtime dependencies are properly loaded,
in case `info` and `load` fail, it fallback to FFmpeg-based implementation.

BC-breaking changes:
 - FFmpeg does not report the number of frames for MP3, this is because MP3 does not store the information of the number of frames. It can be estimated from the audio duration and sample rate, but it might be inaccurate, so we keep it 0.

Depends on
- https://github.com/pytorch/audio/issues/2416
- https://github.com/pytorch/audio/issues/2417
- https://github.com/pytorch/audio/issues/2418
- https://github.com/pytorch/audio/issues/2423
- https://github.com/pytorch/audio/issues/2427

Pull Request resolved: https://github.com/pytorch/audio/pull/2419

Reviewed By: carolineechen

Differential Revision: D36740306

Pulled By: mthrok

fbshipit-source-id: 9e2ad095b8b39e41404970de0d8d9b5aaa856c97

Summary:
Checks download flag and raises error when dataset is missing given download flag exists. Unit tested manually.

edit: Changed path to check as well as comment that is returned.

Pull Request resolved: https://github.com/pytorch/audio/pull/2430

Reviewed By: carolineechen

Differential Revision: D36815729

Pulled By: skim0514

fbshipit-source-id: f062db7919271665b88ec9754d85cfa83b4f6fa3

Summary:
A couple of weeks ago we started to see OpenMP not found error on macOS CI.
From https://github.com/pytorch/audio/issues/2404, we install OpenMP from brew, and build passes, but unit tests are seg-faulting ever since.

https://app.circleci.com/pipelines/github/pytorch/audio/10825/workflows/c0ecae99-d409-4df2-ab91-9bcb126c309d/jobs/671518

The failing test uses `torchaudio.functional.filitfilt`, which uses [OpenMP for parallel execution](https://github.com/pytorch/audio/blob/6057d3cf1c2f3a4c5072a3853a021bb8b4ce61f7/torchaudio/csrc/lfilter.cpp#L20).

This commit reverts https://github.com/pytorch/audio/issues/2404 and disables OpenMP for macOS builds and tests.

Pull Request resolved: https://github.com/pytorch/audio/pull/2431

Reviewed By: atalman

Differential Revision: D36819141

Pulled By: mthrok

fbshipit-source-id: 824300866a55f8b029d21649dc96cd80ae2ff697

Summary:
* Update error messages
* Update audio stream tests

Pull Request resolved: https://github.com/pytorch/audio/pull/2429

Reviewed By: carolineechen, nateanl

Differential Revision: D36812769

Pulled By: mthrok

fbshipit-source-id: 7a51d0c4dbae558010d2e59412333e4a7f00d318

Summary:
Bringing in move seed commit from previous open commit https://github.com/pytorch/audio/issues/2267. Organizes seed to utils.

Pull Request resolved: https://github.com/pytorch/audio/pull/2425

Reviewed By: carolineechen, nateanl

Differential Revision: D36787599

Pulled By: skim0514

fbshipit-source-id: 37a0d632d13d4336a830c4b98bdb04828ed88c20

Summary: Pull Request resolved: https://github.com/pytorch/audio/pull/2426

Reviewed By: nateanl

Differential Revision: D36791423

Pulled By: carolineechen

fbshipit-source-id: e011147a716c940755032b8c68f5717d11fc91bf

Summary: Pull Request resolved: https://github.com/pytorch/audio/pull/2411

Reviewed By: carolineechen

Differential Revision: D36663904

Pulled By: nateanl

fbshipit-source-id: c6a7dd530c9cfbb58b7121ebe02db6ae293cc2d0

Summary:
Move CTC beam search decoder out of prototype to new `torchaudio.models.decoder` module.

hwangjeff mthrok any thoughts on the new module + naming, and if we should move rnnt beam search here as well??

Pull Request resolved: https://github.com/pytorch/audio/pull/2410

Reviewed By: mthrok

Differential Revision: D36784521

Pulled By: carolineechen

fbshipit-source-id: a2ec52f86bba66e03327a9af0c5df8bbefcd67ed

Summary:
Extract from https://github.com/pytorch/audio/issues/2419. Move the `FileObj` definition to dedicated file, so that it can be reused from files other than StreamReader.

Pull Request resolved: https://github.com/pytorch/audio/pull/2427

Reviewed By: carolineechen

Differential Revision: D36794367

Pulled By: mthrok

fbshipit-source-id: 999658f3f4d833566d933c9223e7a5d49d300574

Summary:
Extracted from https://github.com/pytorch/audio/issues/2419. Move the failure of sox_io from C++ to Python layer.

Pull Request resolved: https://github.com/pytorch/audio/pull/2423

Reviewed By: carolineechen

Differential Revision: D36766152

Pulled By: mthrok

fbshipit-source-id: 53f897a608e97b81ebe5df29577374d88ce178f3

Summary:
This PR adds M1 wheel builds for torchaudio
Based on this PR: https://github.com/pytorch/vision/pull/5948
And this Builder [script](https://github.com/pytorch/builder/blob/main/build_m1_domains.sh)

Pull Request resolved: https://github.com/pytorch/audio/pull/2421

Reviewed By: mthrok

Differential Revision: D36767469

Pulled By: atalman

fbshipit-source-id: 9fc3b74b50ee669a230302fd27682702f83f63dc

Summary:
All the unittests jobs are failing due to import error due to protobuf and scipy.
This commit pins the versions of them to an older version.

## protobuf

https://app.circleci.com/pipelines/github/pytorch/audio/10979/workflows/42005226-ca7e-471c-80f4-db09f4bd2089/jobs/692078

```
E   TypeError: Descriptors cannot not be created directly.
E   If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
E   If you cannot immediately regenerate your protos, some other possible workarounds are:
E    1. Downgrade the protobuf package to 3.20.x or lower.
E    2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
E
E   More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
```

https://github.com/protocolbuffers/protobuf/issues/10051
https://github.com/PyTorchLightning/pytorch-lightning/issues/13159

## scipy (pypocketfft)

1.8.1 is causing issue.

https://app.circleci.com/pipelines/github/pytorch/audio/10980/workflows/470a9361-4cc5-4d7c-9264-28fc8b86f1cb/jobs/692267

    ```
    ../env/lib/python3.9/site-packages/librosa/core/audio.py:11: in <module>
        import scipy.signal
    ../env/lib/python3.9/site-packages/scipy/signal/__init__.py:309: in <module>
        from . import _sigtools, windows
    ../env/lib/python3.9/site-packages/scipy/signal/windows/__init__.py:41: in <module>
        from ._windows import *
    ../env/lib/python3.9/site-packages/scipy/signal/windows/_windows.py:7: in <module>
        from scipy import linalg, special, fft as sp_fft
    ../env/lib/python3.9/site-packages/scipy/fft/__init__.py:91: in <module>
        from ._helper import next_fast_len
    ../env/lib/python3.9/site-packages/scipy/fft/_helper.py:3: in <module>
        from ._pocketfft import helper as _helper
    ../env/lib/python3.9/site-packages/scipy/fft/_pocketfft/__init__.py:3: in <module>
        from .basic import *
    ../env/lib/python3.9/site-packages/scipy/fft/_pocketfft/basic.py:6: in <module>
        from . import pypocketfft as pfft
    E   ImportError: /home/circleci/project/env/lib/python3.9/site-packages/torch/lib/../../../.././libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/circleci/project/env/lib/python3.9/site-packages/scipy/fft/_pocketfft/pypocketfft.cpython-39-x86_64-linux-gnu.so)

Pull Request resolved: https://github.com/pytorch/audio/pull/2422

Reviewed By: atalman

Differential Revision: D36764198

Pulled By: mthrok

fbshipit-source-id: 897a79fe9c3165206c2e747147fd0f257fc4f683

Summary:
Add num_frames and bits_per_sample to match with the current
`torchaudio.info` capability.

Pull Request resolved: https://github.com/pytorch/audio/pull/2418

Reviewed By: carolineechen

Differential Revision: D36749077

Pulled By: mthrok

fbshipit-source-id: 7b368ee993cf5ed63ff2f53c9e3b1f50fcce7713

Summary:
Preparation for upcoming change where load/info function will use fallback
if sox_io backend cannot handle the input.

Pull Request resolved: https://github.com/pytorch/audio/pull/2416

Reviewed By: carolineechen

Differential Revision: D36736969

Pulled By: mthrok

fbshipit-source-id: f804cfda3678f13bf0c2f6557a2f82ae42ae3c03

Summary:
Attempt to load ffmpeg extension at the top level import

Preparation to use ffmpeg-based I/O as a fallback for sox_io backend.

Pull Request resolved: https://github.com/pytorch/audio/pull/2417

Reviewed By: carolineechen

Differential Revision: D36736989

Pulled By: mthrok

fbshipit-source-id: 0beb6f459313b5ea91597393ccb12571444c54d9

Summary:
* `Streamer` has been renamed to `StreamReader` when it was moved from prototype to beta.
This commit applies the same name change to the C++ source code.

* Fix miscellaneous lint issues

* Make the code compilable on FFmpeg 5

Pull Request resolved: https://github.com/pytorch/audio/pull/2403

Reviewed By: carolineechen

Differential Revision: D36613053

Pulled By: mthrok

fbshipit-source-id: 69fedd6720d488dadf4dfe7d375ee76d216b215d

Summary:
Follow-up of https://github.com/pytorch/audio/issues/2407, the <script> was not properly closed on pages other than tutorials

Pull Request resolved: https://github.com/pytorch/audio/pull/2409

Reviewed By: carolineechen

Differential Revision: D36632668

Pulled By: mthrok

fbshipit-source-id: 9c0409a8011d77f8689e2dcdc1bd9844d3d31f79

Summary:
This commit fixes multiple issues with documentation.

https://output.circle-artifacts.com/output/job/23245537-e57b-4b9d-9b81-b3df20996d1f/artifacts/0/docs/tutorials/audio_resampling_tutorial.html

1. Duplicated requirejs
The nbsphinx extension introduced in https://github.com/pytorch/audio/pull/2393 pulled a requirejs
which caused the initialization script to halt.
As a result, the right side bar was left uninitialized.

2. Undefined variable error
It turned out that PyTorch's theme expected the downstream projects
to define `collapsedSections` variable.
Currently console log shows `collapsedSections is not defined`.
As a result of this fix, we start to see the + symbol on left side.

3. Fix the behavior of default expand
Tweaks the right-side bar initialization behavior
so that expand-all only happens once, not at every resize.

4. Overwrite the link to GitHub
The `GitHub` tab in main-menu always linked PyTorch core.
This commit adds overwrite to torchaudio page

Pull Request resolved: https://github.com/pytorch/audio/pull/2407

Reviewed By: carolineechen

Differential Revision: D36612904

Pulled By: mthrok

fbshipit-source-id: 56aa7623a8925a241cf4790ac77a87424ad9237c

Summary:
- The multi-channel functions only support complex-valued tensors for spectrogram and PSD matrices.
- The mask can be real-valued or complex-valued, hence there is no explicit assertion for mask.
- The shape of input Tensors need to be verified before the computation. For example, the shape of PSD matrix must be `(..., freq, channel, channel)`, the shape of the mask must be `(..., freq, time)`, etc.
- The autograd unittest of `apply_beamforming` has wrong dimensions for beamform_weights detected by the assertion check. FIx it in this PR.

Pull Request resolved: https://github.com/pytorch/audio/pull/2401

Reviewed By: carolineechen

Differential Revision: D36597689

Pulled By: nateanl

fbshipit-source-id: 6ad1adebe3726851cc1d865650bdf177a98985f6

Summary:
The `LibriLightLimited` dataset is created for fine-tuning SSL models, such as Wav2Vec2 and HuBERT. It is a supervised subset of [Libri-Light](https://github.com/facebookresearch/libri-light) dataset. To distinguish the unsupervised subset and the supervised one, it's clearer to put it in a separate dataset class for fine-tuning purpose.
It contains "10 min", "1 hour", "10 hour" splits.

Pull Request resolved: https://github.com/pytorch/audio/pull/2302

Reviewed By: mthrok

Differential Revision: D36388188

Pulled By: nateanl

fbshipit-source-id: ba49f1c9996be17db5db41127d8ca96224c94249