GitLab

Summary:
Adds unit tests for Emformer RNN-T LibriSpeech recipe. Also makes changes to recipe to resolve errors with pickling lambda functions in Windows.

Pull Request resolved: https://github.com/pytorch/audio/pull/2216

Reviewed By: nateanl

Differential Revision: D34171480

Pulled By: hwangjeff

fbshipit-source-id: 5fcebb457051f3041766324863728411180f5e1e

Summary:
This PR adds the prototype streaming API.
The implementation is based on ffmpeg libraries.

For the detailed usage, please refer to [the resulting tutorial](https://534376-90321822-gh.circle-artifacts.com/0/docs/tutorials/streaming_api_tutorial.html).

Pull Request resolved: https://github.com/pytorch/audio/pull/2164

Reviewed By: hwangjeff

Differential Revision: D33934457

Pulled By: mthrok

fbshipit-source-id: 92ade4aff2d25baf02c0054682d4fbdc9ba8f3fe

Summary:
This commit enables ffmpeg-feature build in tests and
binary builds of all platforms.
(Linux/macOS/Windows x conda/wheel)

It also moves the definition of BUILD_FFMPEG env vars to the
top level `config.yml`.

 ---
Manual checking if all the build log contains `libtorchaudio_ffmpeg`.
### binary build
- [x] `binary_linux_conda_py3.7_cpu`
- [x] `binary_linux_conda_py3.7_cu102`
- [x] `binary_linux_wheel_py3.7_cpu`
- [x] `binary_linux_wheel_py3.7_cu102`
- [x] `binary_macos_conda_py3.7_cpu`
- [x] `binary_macos_wheel_py3.7_cpu`
- [x] `binary_windows_conda_py3.7_cpu`
- [x] `binary_windows_conda_py3.7_cu113`
- [x] `binary_windows_wheel_py3.7_cpu`
- [x] `binary_windows_wheel_py3.7_cu113`

### test
- [x] `unittest_linux_cpu_py3.7`
- [x] `unittest_linux_gpu_py3.7`
- [x] `unittest_macos_cpu_py3.7`
- [x] `unittest_windows_cpu_py3.7`
- [x] `unittest_windows_gpu_py3.7`
- [x] `integration test`

Pull Request resolved: https://github.com/pytorch/audio/pull/2140

Reviewed By: hwangjeff

Differential Revision: D33464430

Pulled By: mthrok

fbshipit-source-id: 2c5b72be75d49019bf1599036180d4e56074e46b

Summary:
Preparation to land Python front-end of ffmpeg-related features.

- Set BUILD_FFMPEG=1 in Linux/macOS unit test jobs
- Install ffmpeg and pkg-config from conda-forge
- Add note about Windows build process
- Temporarily avoid `av_err2str`

Pull Request resolved: https://github.com/pytorch/audio/pull/2114

Reviewed By: hwangjeff

Differential Revision: D33371346

Pulled By: mthrok

fbshipit-source-id: b0e16a35959a49a2166109068f3e0cbbb836e888

Summary:
Similar to https://github.com/pytorch/vision/pull/4788
Make sure the workflow could download right PyTorch with cpu or cuda in case nightly build wasn't ready at that day.

https://app.circleci.com/pipelines/github/pytorch/audio/8427/workflows/11a80738-bcdd-45e3-b37f-328be36c60ee/jobs/438285?invite=true#step-107-542

![image](https://user-images.githubusercontent.com/16190118/142969390-142df7ec-6040-40c1-9a02-17d43f5de05e.png)

Pull Request resolved: https://github.com/pytorch/audio/pull/2026

Reviewed By: hwangjeff, nateanl

Differential Revision: D32634926

Pulled By: mthrok

fbshipit-source-id: 30d6349a0a2ce174b789a5888b1c8e0544a23a37

This commit changes the build process so that the third party code are fetched automatically when `setup.py` is invoked.

This is to allow
1. `build_sdist` to contain the third party library codes, so that source code distribution created at the time release tag is created is complete.
2. It makes it possible to do `pip install https://github.com/pytorch/audio.git`.
    Example:
    ```
    !pip install 'cmake>=3.18' ninja
    !pip install --verbose git+https://github.com/pytorch/audio.git@auto-source
    ```

* Default to BUILD_SOX=1 in non-Windows systems

Since the adaptation of CMake and restricting to the static linking of libsox,
the build process has become much robust with libsox integration enabled.

This commit makes it default behavior to build libsox integration in non-Windows systems.
The build process still checks BUILD_SOX env var so, setting `BUILD_SOX=0` disables it.

* Pin MKL to 2021.2.0

Fixes https://github.com/pytorch/pytorch/issues/62424

* Apply the same  constraint for conda builds

Use less-than-equals constraint to avoid dependency conflicts for
python-3.6 environment

There are some issues on CMake-based integration after 2021-06-19.

```
  Imported target "torch" includes non-existent path
    "/opt/conda/conda-bld/pytorch_1624259172741/work/torch/lib"
  in its INTERFACE_INCLUDE_DIRECTORIES.  Possible reasons include:
  * The path was deleted, renamed, or moved to another location.
  * An install or uninstall procedure did not complete successfully.
  * The installation package was faulty and references files it does not provide.
```

This is being worked on PR like https://github.com/pytorch/pytorch/pull/60403.
The nightly builds are still causing the error, so as a workaround, 
we pin the nightly build version for Linux/macOS unittest.
The issue still happens on Windows unittest and binary builds.

Numpy is a dependency of pytorch so we should let it get resolved as it
install pytorch
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

* Switch to cmake for build
* Hide symbols

* Do not install sox system-wide, so that it will never get mixed up with static one torchaudio builds
* Do not install cmake system-wide
* Move libsndfile installation to Docker image

We have been running unit test with editable installation. (i.e. `python setup.py develop`), with which we missed issues like #842. 

This CC makes installation in CI non-editable, and change test directory structure so that the source code will not shadow the installed version of `torchaudio`. With simple `pytest test`, `pytest` modifies `sys.path` and prepend checked out repository, which shadows the installed version.

To remedy this, the whole test suites has been moved from `./test` to `./test/torchaudio_unittest`. This adds nice module structure to our test code and we can do absolute import in each test module, which makes it possible again to run test with `python -m unittest torchaudio_unittest/XXX.py`

This change does not affect the regular development process (`python setup.py develop` && `pytest test`)

This is a part of PRs to add new "sox_io" backend #726, and depends on #718.

This PR adds `info` function to "sox_io" backend, which allows users to fetch some metadata of an audio file. 
At this moment, the information retrieved are;

 - Number of samples in the audio file
 - Sampling rate
 - Number of channels

* Use env var for switching SoX build and default to not

* Update README

* Fix packaging/CI script

* Clean up extension build mechanism and extension location

* Add back the switch to depend on external sox

* Remove print

* Fix

* Add GPU test

* Run only on master

* Turn on tests when building through conda-build

* skip one failling test on windows.

* Switch to unittest

* Fix conda env activation

* add linux only condition

* Fix librosa installation

* conda-forge channel as a fallback

* Revert "conda-forge channel as a fallback"

This reverts commit 8c15f6638e34792640a3de3e057d9c2995425180.

* Try the pytorch way

* Use conda-forge

* Disable conda-forge

* Install future at setup env

* Move linux unittest CI job tools to dedicated directory

* Restore Linux scripts and move the updated souces to a new subdir

* Add back future

* Remove comments
Co-authored-by: Vincent Quenneville-Belair <vincentqb@gmail.com>
Co-authored-by: moto <855818+mthrok@users.noreply.github.com>

* Add test image with Kaldi executable and third_parties

* Use Ubuntu as bese image since pytorch/many-linux is huge