Skip to content

GitLab

Switch branch/tag
  1. 15 Aug, 2023 1 commit
    • moto's avatar
      [BC-breaking] Update pre-built ffmpeg4 to 4.4.4 (#3561) · bf07ea6b
      moto authored 
      Summary:
In https://github.com/pytorch/audio/pull/3460, we switched the build process for FFmpeg extension.
Since it is complicated to install FFmpeg in some environments, at build time, pre-built binaries and its headers
are downloaded and used as a scaffolding for torchaudio build.

Now even though we did not change any code or FFmpeg version, it turned out that this causes segmentation
fault on Ubuntu when using system Python and FFmpeg 4.4 installed via aptitude.
While investigating the issue, I swapped the said pre-built FFmpeg scaffolding with FFmpeg 4.4 from aptitude,
and the segmentation fault did not happen. This indicates that it is binary compatibility issue.

Before https://github.com/pytorch/audio/issues/3460, each binary build job was building FFmpeg 4.1.8 using the same compiler used to build torchaudio,
but after https://github.com/pytorch/audio/issues/3460 the environments to build FFmpeg 4.1.8 and torchaudio are different. My hypothesis is that
this difference is causing some ABI incompatibility when linking against FFmpeg 4.4. (Also, I don't remember well,
but I read somewhere that 4.4 has a different ABI)

Through experiments, it turned out upgrading the pre-built FFmpeg scaffolding to 4.4 resolves this.
So this commit upgrade the pre-built FFmpeg 4 to 4.4.
The potential (yet unconfirmed) downside is that torchaudio will no longer work with 4.1, 4.2, and 4.3.
Since FFmpeg 4.4 is what Ubuntu 20.04 and 22.04 support by default, and Google Colab is also on 20.04,
I think it is more important to support 4.4.

Therefore we drop the support for 4.1-4.3 from normal build (and official distributions). Those who wish to
use 4.1-4.3 can build torchaudio from source by linking to specific FFmpeg.

Pull Request resolved: https://github.com/pytorch/audio/pull/3561

Reviewed By: hwangjeff

Differential Revision: D48340201

Pulled By: mthrok

fbshipit-source-id: 7ece82910f290c7cf83f58311c4cf6a384e8795e
      bf07ea6b
  3. 27 Jul, 2023 2 commits
  5. 12 Jul, 2023 1 commit
    • moto's avatar
      Support multiple FFmpeg versions (#3464) · 786066b4
      moto authored 
      Summary:
This commit introduces support for multiple FFmpeg versions for OSS binary distributions.

Currently torchaudio only works with FFmpeg 4. This is inconvenient from installing to runtime linking.
This commit allows to pick FFmpeg 4, 5 or 6 at runtime, instead of just looking for v4.

The way it works is that we compile the FFmpeg extension three times with different FFmpeg and ship them.
At runtime, we look for libavutil of specific version and when one is found, load the corresponding FFmpeg extension.
The order of preference is 6, 5, then 4.

To make the build process simple and reproducible, we use pre-built binaries of FFmpeg during the build.
They are LGPL and downloaded from S3 at build time, instead of building every time.

The use of pre-built binaries as scaffolding limits the system that can build torchaudio, so it also introduces
single FFmpeg version support mode. setting FFMPEG_ROOT during the build will change the way binaries are built
so that it will only support one specific version of FFmpeg.

Pull Request resolved: https://github.com/pytorch/audio/pull/3464

Differential Revision: D47300223

Pulled By: mthrok

fbshipit-source-id: 560c7968315e4c8922afa11a4693f648c0356d04
      786066b4
  7. 07 Jul, 2023 1 commit
    • moto's avatar
      Use pre-built binaries for ffmpeg extension (#3460) · f77c3e5b
      moto authored 
      Summary:
This commit changes the way FFmpeg extension is built.

Originally, the build process expected the FFmpeg binaries to be somehow available in build env.
This makes the build process unpredictable and prevents default enabling FFmpeg extension.

The proposed change uses pre-built FFmpeg binaries as build-time only scaffold, which are built in our CI job https://github.com/pytorch/audio/actions/workflows/ffmpeg.yml.

This makes the build process more predictable and removes the necessity to build FFmpeg in our CI.
Currently, it supports macOS (arm64, x86_64), unix (x86_64, aarch64) and windows (amd64).
The downside is that it no longer works with the architecture not listed above.
We can potentially workaround by searching the FFmpeg binaries available in system (the old way) for
these system, but since they are not supported by PyTorch, the priority is low.

Pull Request resolved: https://github.com/pytorch/audio/pull/3460

Differential Revision: D47261885

Pulled By: mthrok

fbshipit-source-id: 223a15e95c9140c95688af968beb35ff40354476
      f77c3e5b
  9. 05 Jul, 2023 1 commit
  11. 13 Jun, 2023 1 commit
  13. 02 Jun, 2023 1 commit
    • moto's avatar
      [BC-Breaking] Remove compute_kaldi_pitch (#3368) · 5bbbb1d5
      moto authored 
      Summary:
This commit removes compute_kaldi_pitch function and the underlying Kaldi integration from torchaudio.

Kaldi pitch function was added in a short period of time by integrating the original Kaldi implementation, instead of reimplementing it in PyTorch.

The Kaldi integration employed a hack which replaces the base vector/matrix implementation of Kaldi with PyTorch Tensor so that there is only one blas library within torchaudio.

Recently, we are making torchaudio more lean, and we don't see a wide adoption of kaldi_pitch feature, so we decided to remove them.

See some of the discussion https://github.com/pytorch/audio/issues/1269

Pull Request resolved: https://github.com/pytorch/audio/pull/3368

Differential Revision: D46406176

Pulled By: mthrok

fbshipit-source-id: ee5e24d825188f379979ddccd680c7323b119b1e
      5bbbb1d5
  15. 01 Jun, 2023 1 commit
  17. 16 May, 2023 1 commit
  19. 28 Apr, 2023 1 commit
    • Yuekai Zhang's avatar
      Add cuctc decoder (#3096) · 0a1801ed
      Yuekai Zhang authored 
      Summary:
This PR implements a CUDA based ctc prefix beam search decoder.

Attach serveral benchmark results using V100 below:
|decoder type| model |datasets       | decoding time (secs)| beam size | batch size | model unit | subsampling times | vocab size |
|--------------|---------|------|-----------------|------------|-------------|------------|-----------------------|------------|
| cuctc |  conformer nemo    |dev clean        |7.68s | 8           |  32       | bpe         |    4  | 1000|
| cuctc |  conformer nemo   |dev clean  (sort by length)      |1.6s | 8           |  32       | bpe         |    4  | 1000|
| cuctc |  wav2vec2.0 torchaudio |dev clean                                |22s | 10           |  1       | char         |    2  | 29|
| cuctc |   conformer espnet   |aishell1 test                             | 5s | 10           |  24       | char         |    4  | 4233|

Note:
1.  The design is to parallel computation through batch and vocab axis, for loop the frames axis. So it's more friendly with smaller sequence lengths, larger vocab size comparing with CPU implementations.
2. WER is the same as CPU implementations. However, it can't decode with LM now.

Resolves: https://github.com/pytorch/audio/issues/2957.

Pull Request resolved: https://github.com/pytorch/audio/pull/3096

Reviewed By: nateanl

Differential Revision: D44709397

Pulled By: mthrok

fbshipit-source-id: 3078c54a2b44dc00eb4a81b4c657487eeff8c155
      0a1801ed
  21. 05 Apr, 2023 1 commit
  23. 29 Dec, 2022 1 commit
  25. 29 Jul, 2022 1 commit
  27. 28 Jul, 2022 1 commit
    • moto's avatar
      Migrate CTC decoder code (#2580) · 39b6343d
      moto authored 
      Summary:
This commit gets rid of our copy of CTC decoder code and
replace it with upstream Flashlight-Text repo.

Pull Request resolved: https://github.com/pytorch/audio/pull/2580

Reviewed By: carolineechen

Differential Revision: D38244906

Pulled By: mthrok

fbshipit-source-id: d274240fc67675552d19ff35e9a363b9b9048721
      39b6343d
  29. 19 Jul, 2022 1 commit
    • moto's avatar
      Remove boost (#2552) · ee631d6b
      moto authored 
      Summary:
After reviewing the code for KenLM it turned out that we can build it without boost.

Pull Request resolved: https://github.com/pytorch/audio/pull/2552

Reviewed By: xiaohui-zhang

Differential Revision: D37949699

Pulled By: mthrok

fbshipit-source-id: 4a4ffae4220d0b764b53f52b93040670d91a84a3
      ee631d6b
  31. 15 Jun, 2022 1 commit
  33. 02 Jun, 2022 1 commit
    • moto's avatar
      Remove mad (#2428) · d2ecba98
      moto authored 
      Summary:
Remove the code related to libmad, which had been disabled in https://github.com/pytorch/audio/issues/2354

In https://github.com/pytorch/audio/issues/2419, we mp3 decoding to ffmpeg. But CI tests were still using libmad.
This commit completely removes libmad from torchaudio.

This is BC-breaking change as `apply_sox_effects_file` function cannot handle MP3, and it cannot fallback to ffmpeg.
The workaround for this is to use `torchaudio.load` then `apply_sox_effects_tensor`.

Pull Request resolved: https://github.com/pytorch/audio/pull/2428

Reviewed By: carolineechen

Differential Revision: D36851805

Pulled By: mthrok

fbshipit-source-id: f98795c59a1ac61cef511f2bbeac37f7c3c69d55
      d2ecba98
  35. 28 Apr, 2022 1 commit
  37. 01 Apr, 2022 1 commit
    • moto's avatar
      Update GNU config files to support `arm64-apple` system (#2307) · 3ed39e15
      moto authored 
      Summary:
This commit
1. Updates the config.guess and config.sub files and
2. applies them to all the third party libraries that use them.

This resolves the following build failure on M1 mac with newer SDK.

On MacBookPro with M1 chip, with the recent OS update, something
about the development environment has been changed (probably newer
version of XCode) and the build stopeed working with the following
errors from third party dependencies.

```
checking build system type... Invalid configuration ‘arm64-apple-darwin20.0.0': machine ‘arm64-apple' not recognized
```

note: config files are taken from https://www.gnu.org/software/gettext/manual/html_node/config_002eguess.html

Pull Request resolved: https://github.com/pytorch/audio/pull/2307

Reviewed By: nateanl

Differential Revision: D35318273

Pulled By: mthrok

fbshipit-source-id: 746ac51dd1816767aa78b88445f76a29acfd29e8
      3ed39e15
  39. 30 Mar, 2022 2 commits
  41. 22 Mar, 2022 1 commit
    • moto's avatar
      Revise the parameterization of third party libraries (#2282) · 7444f568
      moto authored 
      Summary:
Originally, the global property TORCHAUDIO_THIRD_PARTIES was introduced
to handle the optional third party dependencies that can change based on
the build config.

After revising the CMake, it turned out this is not really necessary,
as our torchaudio/csrc/CMakeLists.txt properly branches out for
conditional dependencies. Rather we should leave the global scope untouched.

Pull Request resolved: https://github.com/pytorch/audio/pull/2282

Reviewed By: hwangjeff

Differential Revision: D35059838

Pulled By: mthrok

fbshipit-source-id: ed3557eaa9a669e4466d64893beab5089eca78b8
      7444f568
  43. 06 Mar, 2022 1 commit
  45. 15 Feb, 2022 1 commit
    • moto's avatar
      Improve ffmpeg library discovery (#2204) · 963905e4
      moto authored 
      Summary:
This commit fixes the issue with ffmpeg discovery at build time.
The original implementation had issues like.

1. Wrong usage of FindFFMPEG, which caused mixture of ffmpeg libraries from system directory and user directory.
2. The optional `FFMPEG_ROOT` variable was not set within cmake.

The issue 1 is problematic when a user does not have a permission to
modify the environment. For example, an old version of ffmpeg, which is
installed in a directory managed by the system (such as `/usr/local/lib`),
then there is no way to specify a path in which user installs a supported version
of ffmpeg.

This commit changes the behavior by first searching the library
in `FFMPEG_ROOT` environment variables, then
resorting to the original behavior of searching the custom paths with
system default path.

Also this commirt removes support for `libavresample`, which is deprecated in
ffmpeg 4 and removed in ffmpeg 5.

Pull Request resolved: https://github.com/pytorch/audio/pull/2204

Reviewed By: carolineechen

Differential Revision: D34225769

Pulled By: mthrok

fbshipit-source-id: 95b0bfaaef31e2e69e6df29f789010f48a48210b
      963905e4
  47. 05 Jan, 2022 1 commit
    • moto's avatar
      Update ffmpeg discovery logic (#2124) · d8a65450
      moto authored 
      Summary:
Update ffmpeg discovery logic

Previously the build process used pkg-config to locate
an installation of ffmpeg, which does not work well Windows/CentOS.

This commit update the discovery process to use the custom
FindFFMPEG.cmake adopted from Kitware/VTK repository with addition of
conda environment.

 The custom discovery logic can support Windows and CentOS.

Pull Request resolved: https://github.com/pytorch/audio/pull/2124

Reviewed By: carolineechen

Differential Revision: D33429564

Pulled By: mthrok

fbshipit-source-id: 6cb50c1d8c58f51e0f3f3af5c5b541aa3a699bba
      d8a65450
  49. 04 Jan, 2022 1 commit
    • moto's avatar
      [CI] Install tools from conda instead of brew (#1873) · df0175e8
      moto authored 
      Summary:
Currently, macOS CI jobs install `pkg-config` and `wget` with `brew`.
This is problematic as brew takes a long time with auto-update, and disabling the auto-update is not an ideal solution.
Conda also distributes these packages, so switching to conda.

Example issues with brew installation.
https://app.circleci.com/pipelines/github/pytorch/audio/7825/workflows/53965bcf-6ddf-4e42-ad52-83fd1bbab717

This commit removes the use of `brew` by
1. Replacing the use of `wget` with `curl` (pre-installed in most distro)
2. Install `pkg-condig` from conda.
    Note: All the macOS jobs, including binary build jobs, uses conda. Using `pkg-config` from Conda makes it easy to discover the packages installed from conda. (like `ffmpeg` in https://github.com/pytorch/audio/issues/2122)
3. Add `pkg-config` to conda build-time dependency
4. Make sure that the availability of `pkg-config` is explicitly checked when `sox` is being configured. (otherwise, it will fail at somewhere in the middle of build process with somewhat unintuitve error message)

Pull Request resolved: https://github.com/pytorch/audio/pull/1873

Reviewed By: carolineechen, nateanl

Differential Revision: D33404975

Pulled By: mthrok

fbshipit-source-id: ae512d3a3a422ebfe3b46c492bed44deecc36e72
      df0175e8
  51. 30 Dec, 2021 3 commits
    • moto's avatar
      Build ffmpeg-features in Linux/macOS unittests (#2114) · 9f14fa63
      moto authored 
      Summary:
Preparation to land Python front-end of ffmpeg-related features.

- Set BUILD_FFMPEG=1 in Linux/macOS unit test jobs
- Install ffmpeg and pkg-config from conda-forge
- Add note about Windows build process
- Temporarily avoid `av_err2str`

Pull Request resolved: https://github.com/pytorch/audio/pull/2114

Reviewed By: hwangjeff

Differential Revision: D33371346

Pulled By: mthrok

fbshipit-source-id: b0e16a35959a49a2166109068f3e0cbbb836e888
      9f14fa63
    • Joao Gomes's avatar
      Enforce lint checks and fix/mute lint errors (#2116) · 8ed14782
      Joao Gomes authored 
      Summary:
cc mthrok

Pull Request resolved: https://github.com/pytorch/audio/pull/2116

Reviewed By: mthrok

Differential Revision: D33368453

Pulled By: jdsgomes

fbshipit-source-id: 09cf3fe5ed6f771c2f16505633c0e59b0c27453c
      8ed14782
    • moto's avatar
      Add a switch to build ffmpeg binding (#2048) · ece03edc
      moto authored 
      Summary:
This PR adds `BUILD_FFMPEG` switch to torchaudio build process so that features related to ffmpeg are built.
The flag is false by default, so no CI jobs or development flow are affected.

This is because handling the dependencies around ffmpeg is a bit tricky.
Currently, the CMake file uses `pkg-config` to find an ffmpeg installation in the system.
This works fine for both conda-based installation and system-managed installation (like `apt`).

In subsequent PRs, I will find a solution that works for local development and binary distributions.

Pull Request resolved: https://github.com/pytorch/audio/pull/2048

Reviewed By: hwangjeff, nateanl

Differential Revision: D33367260

Pulled By: mthrok

fbshipit-source-id: 94517acecb62bd6d4e96d4b7cbc3ab3c2a25706c
      ece03edc
  53. 20 Dec, 2021 2 commits
  55. 18 Dec, 2021 1 commit
  57. 17 Dec, 2021 1 commit
    • moto's avatar
      Add static build of KenLM (#2076) · adc559a8
      moto authored 
      Summary:
Add KenLM and its dependencies required for static build (`zlib`, `bzip2`, `lzma` and `boost-thread`).

The KenLM and its dependencies are build but since no corresponding code on torchaudio side is changed, the resulting torchaudio extension module is not changed. (therefore, as long as build process passes on CI this PR should be good to go.)

Pull Request resolved: https://github.com/pytorch/audio/pull/2076

Reviewed By: carolineechen

Differential Revision: D33189980

Pulled By: mthrok

fbshipit-source-id: 6096113128b939f3cf70990c99aacc4aaa954584
      adc559a8
  59. 31 Aug, 2021 1 commit
    • Nikita Shulga's avatar
      Fix CUDA build logic for _torchaudio.so (#1737) · e3c082b7
      Nikita Shulga authored 
      It's wrong to depend on `${TORCH_LIBRARIES}` as it pulls in explicit
`libcuda.so.1` dependency, which violates the assumption that GPU
accelerated libraries should be loadable with no NVIDIA drivers installed

Instead, make it depend on `torch` target, which includes all necessary
Torch C++ API dependences
      e3c082b7
  61. 18 Aug, 2021 1 commit
    • moto's avatar
      Guard Kaldi's version generation (#1715) · df9d0b47
      moto authored 
      When building torchaudio from source, `get_version.sh` from kaldi is executed everytime,
which results in kaldi-bindings to be always rebuilt.

This commit add "if" guard to the part so that they are not always executed.
      df9d0b47
  63. 28 Jun, 2021 1 commit
  65. 25 May, 2021 1 commit
  67. 30 Apr, 2021 1 commit
  69. 19 Apr, 2021 1 commit