GitLab

Summary:
The version of MKL that is installed alongside PyTorch has been bumped
to 2022.1 on Windows and it is causing installation issue in unit tests.

This commit pins the previous version

Backport of https://github.com/pytorch/audio/issues/2463

Pull Request resolved: https://github.com/pytorch/audio/pull/2486

Reviewed By: nateanl

Differential Revision: D37160156

Pulled By: mthrok

fbshipit-source-id: 7e3a30c25782b349a3cad2ee6d1141affc921881

Summary:
All the unittests jobs are failing due to import error due to protobuf and scipy.
This commit pins the versions of them to an older version.

## protobuf

https://app.circleci.com/pipelines/github/pytorch/audio/10979/workflows/42005226-ca7e-471c-80f4-db09f4bd2089/jobs/692078

```
E   TypeError: Descriptors cannot not be created directly.
E   If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
E   If you cannot immediately regenerate your protos, some other possible workarounds are:
E    1. Downgrade the protobuf package to 3.20.x or lower.
E    2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
E
E   More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
```

https://github.com/protocolbuffers/protobuf/issues/10051
https://github.com/PyTorchLightning/pytorch-lightning/issues/13159

## scipy (pypocketfft)

1.8.1 is causing issue.

https://app.circleci.com/pipelines/github/pytorch/audio/10980/workflows/470a9361-4cc5-4d7c-9264-28fc8b86f1cb/jobs/692267

    ```
    ../env/lib/python3.9/site-packages/librosa/core/audio.py:11: in <module>
        import scipy.signal
    ../env/lib/python3.9/site-packages/scipy/signal/__init__.py:309: in <module>
        from . import _sigtools, windows
    ../env/lib/python3.9/site-packages/scipy/signal/windows/__init__.py:41: in <module>
        from ._windows import *
    ../env/lib/python3.9/site-packages/scipy/signal/windows/_windows.py:7: in <module>
        from scipy import linalg, special, fft as sp_fft
    ../env/lib/python3.9/site-packages/scipy/fft/__init__.py:91: in <module>
        from ._helper import next_fast_len
    ../env/lib/python3.9/site-packages/scipy/fft/_helper.py:3: in <module>
        from ._pocketfft import helper as _helper
    ../env/lib/python3.9/site-packages/scipy/fft/_pocketfft/__init__.py:3: in <module>
        from .basic import *
    ../env/lib/python3.9/site-packages/scipy/fft/_pocketfft/basic.py:6: in <module>
        from . import pypocketfft as pfft
    E   ImportError: /home/circleci/project/env/lib/python3.9/site-packages/torch/lib/../../../.././libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/circleci/project/env/lib/python3.9/site-packages/scipy/fft/_pocketfft/pypocketfft.cpython-39-x86_64-linux-gnu.so)

Pull Request resolved: https://github.com/pytorch/audio/pull/2422

Reviewed By: atalman

Differential Revision: D36764198

Pulled By: mthrok

fbshipit-source-id: 897a79fe9c3165206c2e747147fd0f257fc4f683

Summary:
Make them more aligned with ones in
https://github.com/pytorch/vision/blob/main/.circleci/unittest/linux/scripts/setup_env.sh

This is preliminary step towards eradicating unneeded conda-forge dependencies, see https://github.com/pytorch/audio/pull/2260

Pull Request resolved: https://github.com/pytorch/audio/pull/2265

Reviewed By: mthrok

Differential Revision: D34499635

Pulled By: malfet

fbshipit-source-id: f87a3e4568aeeab9c6787a777c3231153c4539f0

Summary:
Adding py3.10 to audio

Pull Request resolved: https://github.com/pytorch/audio/pull/2224

Reviewed By: malfet, atalman, mthrok

Differential Revision: D34442377

Pulled By: seemethere

fbshipit-source-id: 2656de73427063958d609a74c01b526a476cb06a

Summary:
Adds unit tests for Emformer RNN-T LibriSpeech recipe. Also makes changes to recipe to resolve errors with pickling lambda functions in Windows.

Pull Request resolved: https://github.com/pytorch/audio/pull/2216

Reviewed By: nateanl

Differential Revision: D34171480

Pulled By: hwangjeff

fbshipit-source-id: 5fcebb457051f3041766324863728411180f5e1e

Summary:
This PR adds the prototype streaming API.
The implementation is based on ffmpeg libraries.

For the detailed usage, please refer to [the resulting tutorial](https://534376-90321822-gh.circle-artifacts.com/0/docs/tutorials/streaming_api_tutorial.html).

Pull Request resolved: https://github.com/pytorch/audio/pull/2164

Reviewed By: hwangjeff

Differential Revision: D33934457

Pulled By: mthrok

fbshipit-source-id: 92ade4aff2d25baf02c0054682d4fbdc9ba8f3fe

Summary:
This commit enables ffmpeg-feature build in tests and
binary builds of all platforms.
(Linux/macOS/Windows x conda/wheel)

It also moves the definition of BUILD_FFMPEG env vars to the
top level `config.yml`.

 ---
Manual checking if all the build log contains `libtorchaudio_ffmpeg`.
### binary build
- [x] `binary_linux_conda_py3.7_cpu`
- [x] `binary_linux_conda_py3.7_cu102`
- [x] `binary_linux_wheel_py3.7_cpu`
- [x] `binary_linux_wheel_py3.7_cu102`
- [x] `binary_macos_conda_py3.7_cpu`
- [x] `binary_macos_wheel_py3.7_cpu`
- [x] `binary_windows_conda_py3.7_cpu`
- [x] `binary_windows_conda_py3.7_cu113`
- [x] `binary_windows_wheel_py3.7_cpu`
- [x] `binary_windows_wheel_py3.7_cu113`

### test
- [x] `unittest_linux_cpu_py3.7`
- [x] `unittest_linux_gpu_py3.7`
- [x] `unittest_macos_cpu_py3.7`
- [x] `unittest_windows_cpu_py3.7`
- [x] `unittest_windows_gpu_py3.7`
- [x] `integration test`

Pull Request resolved: https://github.com/pytorch/audio/pull/2140

Reviewed By: hwangjeff

Differential Revision: D33464430

Pulled By: mthrok

fbshipit-source-id: 2c5b72be75d49019bf1599036180d4e56074e46b

Summary:
Update the internal of `skipIfXXX` decorators so that tests in CI will not be automatically skipped.

Currently we automatically skip some tests based on the availability of related features/test tools.
This causes issues where we miss signals on certain important features. (CUDA on Windows) https://github.com/pytorch/audio/issues/1565

The new `skipIf` decorator will fail if in CI unless it is explicitly allowed to skip tests.
It does so by checking `CI` and `TORCHAUDIO_TEST_ALLOW_SKIP_IF_XXX` environment variables.

For non-CI environments, the behavior is same as before, but users can now set `TORCHAUDIO_TEST_ALLOW_SKIP_IF_XXX=false` to disallow the automatic skip.

Results without `TORCHAUDIO_TEST_ALLOW_SKIP_IF_XXX` https://app.circleci.com/pipelines/github/pytorch/audio/9112/workflows/4e6db046-a1a2-4965-b0fe-d5baf4a1efac

Pull Request resolved: https://github.com/pytorch/audio/pull/2127

Reviewed By: hwangjeff

Differential Revision: D33430711

Pulled By: mthrok

fbshipit-source-id: d8954dd720469c5ab0f34ea062fd8cf04a8afa3e

Summary:
The unit test failures seems to be caused by [conda 4.11](https://github.com/conda/conda/issues/11096)
Remove conda update line fixes the issue.

Pull Request resolved: https://github.com/pytorch/audio/pull/2069

Reviewed By: carolineechen

Differential Revision: D33023851

Pulled By: nateanl

fbshipit-source-id: 73246189d4ccc541e366a5367f532a5b456af8f8

Summary:
Similar to https://github.com/pytorch/vision/pull/4788
Make sure the workflow could download right PyTorch with cpu or cuda in case nightly build wasn't ready at that day.

https://app.circleci.com/pipelines/github/pytorch/audio/8427/workflows/11a80738-bcdd-45e3-b37f-328be36c60ee/jobs/438285?invite=true#step-107-542

![image](https://user-images.githubusercontent.com/16190118/142969390-142df7ec-6040-40c1-9a02-17d43f5de05e.png)

Pull Request resolved: https://github.com/pytorch/audio/pull/2026

Reviewed By: hwangjeff, nateanl

Differential Revision: D32634926

Pulled By: mthrok

fbshipit-source-id: 30d6349a0a2ce174b789a5888b1c8e0544a23a37

This commit changes the build process so that the third party code are fetched automatically when `setup.py` is invoked.

This is to allow
1. `build_sdist` to contain the third party library codes, so that source code distribution created at the time release tag is created is complete.
2. It makes it possible to do `pip install https://github.com/pytorch/audio.git`.
    Example:
    ```
    !pip install 'cmake>=3.18' ninja
    !pip install --verbose git+https://github.com/pytorch/audio.git@auto-source
    ```

* enable windows cudatests

* add this dir

* minor change

* vs integration

* Update cuda_install.bat

* add logs

* minor change

* minor change

* cp vision conda activate

* mv vc_env_helper.bat

* minor change

* exit if cuda not avaiable

* install numpy

* improt CMakeLists

* check cuda

* minor change

* change windows GPU image from previous to stable

* set libtorch audio suffix as pyd on Windows

* reduce changes

* check env settings

No longer needed for Python 3.9
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Numpy is a dependency of pytorch so we should let it get resolved as it
install pytorch
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

librosa 0.8.0 has been released which include fix for
https://github.com/librosa/librosa/issues/1160 at
https://github.com/librosa/librosa/pull/1164

We have been running unit test with editable installation. (i.e. `python setup.py develop`), with which we missed issues like #842. 

This CC makes installation in CI non-editable, and change test directory structure so that the source code will not shadow the installed version of `torchaudio`. With simple `pytest test`, `pytest` modifies `sys.path` and prepend checked out repository, which shadows the installed version.

To remedy this, the whole test suites has been moved from `./test` to `./test/torchaudio_unittest`. This adds nice module structure to our test code and we can do absolute import in each test module, which makes it possible again to run test with `python -m unittest torchaudio_unittest/XXX.py`

This change does not affect the regular development process (`python setup.py develop` && `pytest test`)

* Migrate fbank tests

* Update CCI job environment

* Remove invalid test cases
Signed-off-by: Bhargav Kathivarapu <bhargavkathivarapu31@gmail.com>

* Use env var for switching SoX build and default to not

* Update README

* Fix packaging/CI script

* Enable CUDA tests for Windows

* Add back branch filters

* Turn on tests when building through conda-build

* skip one failling test on windows.

* Switch to unittest

* Fix conda env activation

* add linux only condition

* Fix librosa installation

* conda-forge channel as a fallback

* Revert "conda-forge channel as a fallback"

This reverts commit 8c15f6638e34792640a3de3e057d9c2995425180.

* Try the pytorch way

* Use conda-forge

* Disable conda-forge

* Install future at setup env

* Move linux unittest CI job tools to dedicated directory

* Restore Linux scripts and move the updated souces to a new subdir

* Add back future

* Remove comments
Co-authored-by: Vincent Quenneville-Belair <vincentqb@gmail.com>
Co-authored-by: moto <855818+mthrok@users.noreply.github.com>