GitLab

Summary:
With the introduction of tutorials, the turn around time for doc build
has become longer. By default, the tutorial is not built but SPHINXOPT=-W
treats it as error.

This commit disable the option for the local build while keeping it
for the CI.

Pull Request resolved: https://github.com/pytorch/audio/pull/2013

Reviewed By: carolineechen

Differential Revision: D32538952

Pulled By: mthrok

fbshipit-source-id: eae4ffd87100dff466f91abfe26a82aa702d605a

With the introduction of TTS tutorial (#1973), it takes more than couple of minutes
to build documentation. This commit makes the doc build process defaults to
not build tutorials.

To build tutorials one can use environment variable `BUILD_GALLERY=1`,
and set `GALLERY_PATTERN=...` to filter the tutorials to build.

This `GALLERY_PATTERN` is same approach as in `tutorials` repo.

https://github.com/pytorch/tutorials/blob/cbf2238df0e78d84c15bd94288966d2f4b2e83ae/conf.py#L75-L83

Also this commit dynamically parse the subdirectory of `examples/gallery` so that when a new category of examples are added, it will automatically parsed.

This commit changes the build process so that the third party code are fetched automatically when `setup.py` is invoked.

This is to allow
1. `build_sdist` to contain the third party library codes, so that source code distribution created at the time release tag is created is complete.
2. It makes it possible to do `pip install https://github.com/pytorch/audio.git`.
    Example:
    ```
    !pip install 'cmake>=3.18' ninja
    !pip install --verbose git+https://github.com/pytorch/audio.git@auto-source
    ```

* set cu113 for unittest_windows_gpu

* fix old logic

* Update .circleci/regenerate.py
Co-authored-by: Nikita Shulga <nikita.shulga@gmail.com>

* Limit Windows GPU testing to CUDA-11.3 only

Which is the only CUDA version that planned to be supported on Windows
for the upcoming release

* Move unittests to 11.3 as well

* enable windows cudatests

* add this dir

* minor change

* vs integration

* Update cuda_install.bat

* add logs

* minor change

* minor change

* cp vision conda activate

* mv vc_env_helper.bat

* minor change

* exit if cuda not avaiable

* install numpy

* improt CMakeLists

* check cuda

* minor change

* change windows GPU image from previous to stable

* set libtorch audio suffix as pyd on Windows

* reduce changes

* check env settings

CCI says circleci/python:3.8 is deprecaetd. Migrating to cimg/python:3.8.

> CircleCI’s latest pre-built container images were designed from the ground up to help your team build more reliably.
> Our new images were built specifically for continuous integration projects and they are our most deterministic, performant, and efficient images yet.

> As of Dec 31, 2021, legacy images will no longer be supported on CircleCI.

Related links

- https://circleci.com/blog/announcing-our-next-generation-convenience-images-smaller-faster-more-deterministic
- https://discuss.circleci.com/t/legacy-convenience-image-deprecation/41034
- https://circleci.com/docs/2.0/next-gen-migration-guide

Cherry-pick of
https://github.com/pytorch/vision/commit/d2460a75de237cfef8e5c3415f7bb0ad8467c0e5
into this repo

Fixes https://github.com/pytorch/audio/issues/1750

Downloading CUDA-11.1 from conda-forge takes a while

* Remove some obsolete conditions about CUDA-10.0 from `pkg_helpers.bash`.
* Use `USE_CUDA` instead of `FORCE_CUDA` in `pkg_helpers.bash`
* Do not define `NO_CUDA_PACKAGE` in build_wheel.sh and build_cuda.sh
* Add conda-forge for Win cuda-11.1 builds
* Pass USE_CUDA / TORCH_CUDA_ARCH_LIST to conda build
* Add selected CUDA to path
* Don't define USE_CUDA for ROCM

TODO: Fix Windows CUDA builds

Copy `cuda_install.bat` from
https://github.com/pytorch/vision/blob/3d9d264198e2167030dc04bf453aeca47daa37db/packaging/windows/internal/cuda_install.bat

Add `windows_intall_cuda` step and include it in `binary_windows_conda`
and `binary_windows_wheel` workflows

* Default to BUILD_SOX=1 in non-Windows systems

Since the adaptation of CMake and restricting to the static linking of libsox,
the build process has become much robust with libsox integration enabled.

This commit makes it default behavior to build libsox integration in non-Windows systems.
The build process still checks BUILD_SOX env var so, setting `BUILD_SOX=0` disables it.

* Pin MKL to 2021.2.0

Fixes https://github.com/pytorch/pytorch/issues/62424

* Apply the same  constraint for conda builds

Use less-than-equals constraint to avoid dependency conflicts for
python-3.6 environment

No longer needed for Python 3.9
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

There are some issues on CMake-based integration after 2021-06-19.

```
  Imported target "torch" includes non-existent path
    "/opt/conda/conda-bld/pytorch_1624259172741/work/torch/lib"
  in its INTERFACE_INCLUDE_DIRECTORIES.  Possible reasons include:
  * The path was deleted, renamed, or moved to another location.
  * An install or uninstall procedure did not complete successfully.
  * The installation package was faulty and references files it does not provide.
```

This is being worked on PR like https://github.com/pytorch/pytorch/pull/60403.
The nightly builds are still causing the error, so as a workaround, 
we pin the nightly build version for Linux/macOS unittest.
The issue still happens on Windows unittest and binary builds.

* add rocm binary

use the term cu_version

update pkg_helpers

add docker wheel option

update config yml

add force_cuda option

* update config.yml

* remove conda builds

* Add CUDA binary builds

* Add "cuda_version": "cpu" to doc build jobs

* Add required cu_versions to pkg_helpers

* Regenerate config.yml

All macOS CI jobs were failing at `brew`.
The issue was the macOS version `10.13`, which is no longer supported by `brew`.
The macOS version is indirectly controlled by `xcode` version as listed [here](https://circleci.com/docs/2.0/testing-ios/#supported-xcode-versions).

Updating the `xcode` to `"12.0"` as in PyTorch core bump macOS version to `10.15` and it resolves the issue.
`Python 3.9` is still failing but it's packaging issue on upstream, which needs to be resolved separately.