GitLab

Summary:
Add KenLM and its dependencies required for static build (`zlib`, `bzip2`, `lzma` and `boost-thread`).

The KenLM and its dependencies are build but since no corresponding code on torchaudio side is changed, the resulting torchaudio extension module is not changed. (therefore, as long as build process passes on CI this PR should be good to go.)

Pull Request resolved: https://github.com/pytorch/audio/pull/2076

Reviewed By: carolineechen

Differential Revision: D33189980

Pulled By: mthrok

fbshipit-source-id: 6096113128b939f3cf70990c99aacc4aaa954584

This commit changes the build process so that the third party code are fetched automatically when `setup.py` is invoked.

This is to allow
1. `build_sdist` to contain the third party library codes, so that source code distribution created at the time release tag is created is complete.
2. It makes it possible to do `pip install https://github.com/pytorch/audio.git`.
    Example:
    ```
    !pip install 'cmake>=3.18' ninja
    !pip install --verbose git+https://github.com/pytorch/audio.git@auto-source
    ```

Also update README.md to mention 1.9.1

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

* Switch to cmake for build
* Hide symbols

Currently `setuptools` assumes that torhcaudio is zip safe and performs
egg (zip) installation when `python setup.py install` (or `pip install .`).

When `torchaudio` loads extension module written with Torchscript, 
the corresponding loading function requires the `so` file to be present as 
an actual file, and this does not work well with egg installation.

This CC fixes this by setting `zip_safe=False` in setup.py and disable egg
installation, so that `python setup.py install` installs uncompressed files
with regular directory structure.

setuptools warns that it should be imported before Distutils

```
site-packages/setuptools/distutils_patch.py:25: UserWarning: Distutils was imported before Setuptools. This usage is discouraged and may exhibit undesirable behaviors or errors. Please use Setuptools' objects directly or at least import Setuptools first.
```

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

* Clean up extension build mechanism and extension location

* Add back the switch to depend on external sox

* Remove print

* Fix

This PR
 - Changes the location of C++ codes from
     - from `torchaudio/torch_sox.cpp` to `torchaudio/csrc/sox.cpp`
     - from `torchaudio/torch_sox.h` to `torchaudio/csrc/sox.h`
 - Changes the location where the resulting library is placed
     - from `_torch_sox.so` (outside of `torchaudio` module)
        to `torchaudio/_torchaudio.so`.

Fix #628

* [WIP] Add Windows CI

* Remove cu_version

* checkout_merge -> checkout

* Add build script

* Switch backend to soundfile

* Remove soundfile as dependency

* Rename jobs

* Fix lint

* New entry points are packaging/build_wheel.sh and packaging/build_conda.sh. The only mandatory environment variable you have to set is PYTHON_VERSION
* CircleCI configuration uses 2.1-style parametrized builds to let you toggle python version, etc. as you do builds. We create a separate job per build configuration for maximum parallelism
* build_tools/packaging got moved to packaging, to be in-line with directory structure in torchvision
* The build_conda.sh and build_wheel.sh delegate most of the heavy lifting to pkg_helpers.bash, which defines a number of bash functions for performing common operations. The intent is that I'll copy-paste this file between other domain API projects.
* TORCHAUDIO_ prefix removed from envvars, so that I can more easily share packaging scripts between projects
* BUILD_VERSION is completely gone; just change the version number if you need to rebuild
* No more logic for cloning and checking out a fresh copy of torchaudio

Billing of changes:

* Can now toggle what branch is built using BRANCH envvar
* PyTorch nightly we were built against is hardcoded into conda meta.yaml
* Disable Conda Python 3.5 builds as we don't have nightlies for torch at the moment
* No more torchaudio-cpu or torchaudio-nightly suffix
* Remove librosa from testing stanza, so that conda build works with stock anaconda channel
* Change convention on PREFIX to be what we previously called $PREFIX/audio
* Make build_from_source.sh cd-neutral

Signed-off-by: Edward Z. Yang ezyang@fb.com

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

Signed-off-by: Edward Z. Yang <ezyang@fb.com>

This reverts commit cfe3f296.

* Revert "Revert "Rewrote C code with C++ extensions" (#39)"

This reverts commit 7236e3f4

.

* Fix import error of C extension
Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Revert "Nits"

This reverts commit 2db284c4.

* Revert "Conform to sox library a bit better"

This reverts commit c340a8d1.

* Revert "Added test to make sure loading and saving gives the same file"

This reverts commit cba11009.

* Revert "Rewrote C code with C++ extensions"

This reverts commit 8a41ecdc.