Skip to content

GitLab

Switch branch/tag
  1. 05 May, 2023 1 commit
    • moto's avatar
      Fix MKL issue on Intel mac build (#3307) · 3e897ca7
      moto authored 
      Summary:
* Remove MKL and NumPy from Conda build env
* Remove `caffe2::mkl` dependency from `torch_cpu`, which introduced unnecessary and undesided dependency on Intel mac.

TorchAudio does not use BLAS libraries directly, thus all the mentions to MKL should be removed from the codebase.
However, this was causing an issue on Intel mac. It turned out that `torch_cpu` target is pulling `caffe2::mkl` dependency, and the linker on macOS keeps library dependency even if no symbol from that library is used. This stray mkl dependency should be fixed on core side, but also we can modify the target temporarily and remove them.

Also we don't need NumPy on build/run time, so that is removed as well.

Pull Request resolved: https://github.com/pytorch/audio/pull/3307

Reviewed By: atalman

Differential Revision: D45606944

Pulled By: mthrok

fbshipit-source-id: 853411ccbbca31796b808a2b052b4cfa564718cd
      3e897ca7
  3. 04 May, 2023 3 commits
    • atalman's avatar
      Add older mkl build contraint only (#3302) · 1e48af06
      atalman authored 
      Summary:
Similar to what we used to have here:
https://github.com/pytorch/test-infra/pull/3896/files

Pull Request resolved: https://github.com/pytorch/audio/pull/3302

Reviewed By: nateanl

Differential Revision: D45574845

Pulled By: atalman

fbshipit-source-id: 142c35dfd811a5f5c170dcd082bec8d055edd9cb
      1e48af06
    • atalman's avatar
      Add mkl dependency to torchaudio MacOS x86 builds (#3300) · b5795943
      atalman authored 
      Summary:
Add mkl dependency to torchaudio MacOS x86 builds

Already tested here: https://github.com/pytorch/audio/actions/runs/4878179835/jobs/8703586137

Pull Request resolved: https://github.com/pytorch/audio/pull/3300

Reviewed By: jeanschmidt, mthrok

Differential Revision: D45566352

Pulled By: atalman

fbshipit-source-id: a0376016506891240b2dd03d4fa4889028bf764b
      b5795943
    • Xiaohui Zhang's avatar
      Extend mask_along_axis{,_iid} (#3289) · 74bd971a
      Xiaohui Zhang authored 
      Summary:
(1/2 of the previous [PR](https://github.com/pytorch/audio/pull/2360) which I accidentally closed)

The previous way of doing SpecAugment via Frequency/TimeMasking transforms has the following problems:
- Only zero masking can be done; masking by mean value is not supported.
- mask_along_axis is hard-coded to mask the 1st dimension and mask_along_axis_iid is hard-code to mask the 2nd or 3rd dimension of the input tensor.
- For 3D spectrogram tensors where the first dimension is batch or channel, features from the same batch or different channels have to use the same mask, because mask_along_axis_iid only support 4D tensors, because of the above hard-coding
- For 2D spectrogram tensors w/o a batch or channel dimension, Time/Frequency masking can't be applied at all, since mask_along_axis only support 3D tensors, because of the above hard-coding.
- It's not straightforward to apply multiple time/frequency masks by the current design.

To solve these issues, here we
- Extend mask_along_axis_iid to support 3D tensors and mask_along_axis to support 2D tensors. Now both of them are able to mask one of the last two dimensions (where the time or frequency dimension lives) of the input tensor.

The introduction of SpecAugment transform will be done in another PR.

Pull Request resolved: https://github.com/pytorch/audio/pull/3289

Reviewed By: hwangjeff

Differential Revision: D45460357

Pulled By: xiaohui-zhang

fbshipit-source-id: 91bf448294799f13789d96a13d4bae2451461ef3
      74bd971a
  5. 03 May, 2023 4 commits
  7. 02 May, 2023 3 commits
  9. 01 May, 2023 2 commits
  11. 29 Apr, 2023 1 commit
  13. 28 Apr, 2023 1 commit
    • Yuekai Zhang's avatar
      Add cuctc decoder (#3096) · 0a1801ed
      Yuekai Zhang authored 
      Summary:
This PR implements a CUDA based ctc prefix beam search decoder.

Attach serveral benchmark results using V100 below:
|decoder type| model |datasets       | decoding time (secs)| beam size | batch size | model unit | subsampling times | vocab size |
|--------------|---------|------|-----------------|------------|-------------|------------|-----------------------|------------|
| cuctc |  conformer nemo    |dev clean        |7.68s | 8           |  32       | bpe         |    4  | 1000|
| cuctc |  conformer nemo   |dev clean  (sort by length)      |1.6s | 8           |  32       | bpe         |    4  | 1000|
| cuctc |  wav2vec2.0 torchaudio |dev clean                                |22s | 10           |  1       | char         |    2  | 29|
| cuctc |   conformer espnet   |aishell1 test                             | 5s | 10           |  24       | char         |    4  | 4233|

Note:
1.  The design is to parallel computation through batch and vocab axis, for loop the frames axis. So it's more friendly with smaller sequence lengths, larger vocab size comparing with CPU implementations.
2. WER is the same as CPU implementations. However, it can't decode with LM now.

Resolves: https://github.com/pytorch/audio/issues/2957.

Pull Request resolved: https://github.com/pytorch/audio/pull/3096

Reviewed By: nateanl

Differential Revision: D44709397

Pulled By: mthrok

fbshipit-source-id: 3078c54a2b44dc00eb4a81b4c657487eeff8c155
      0a1801ed
  15. 25 Apr, 2023 1 commit
  17. 19 Apr, 2023 2 commits
  19. 18 Apr, 2023 1 commit
  21. 12 Apr, 2023 3 commits
  23. 11 Apr, 2023 2 commits
  25. 10 Apr, 2023 4 commits
  27. 07 Apr, 2023 5 commits
  29. 06 Apr, 2023 2 commits
    • moto's avatar
      Remove custom flashlight import (#3246) · ae614ed3
      moto authored 
      Summary:
In https://github.com/pytorch/audio/pull/3232, the CTC decoder is excluded from binary distribution.
To use CTCDecoder, users need to install flashlight-text.

Currently, if flashlight-text is not available, torchaudio still attempts to import the custom bundle.
This commit clean up this behavior by delaying the error until one of the components is actually used,
and providing a better message.

Pull Request resolved: https://github.com/pytorch/audio/pull/3246

Test Plan:
Binary smoke tests import torchaudio without installing flashlight.
Unit test CI jobs run the CTC decoder with flashlight installed.

Reviewed By: jacobkahn

Differential Revision: D44748413

Pulled By: mthrok

fbshipit-source-id: 21d2cbd9961ed88405a739cc682071066712f5e4
      ae614ed3
    • Jeff Hwang's avatar
      Add frame writing API to StreamWriter (#3244) · f4d94cab
      Jeff Hwang authored 
      Summary:
Pull Request resolved: https://github.com/pytorch/audio/pull/3244

Adds methods to `StreamWriter` that allow for passing in `AVFrame` instances rather than tensors.

Reviewed By: mthrok

Differential Revision: D44589256

fbshipit-source-id: f100e0d349708482b873a9a4bae1eaf5eb65301a
      f4d94cab
  31. 05 Apr, 2023 2 commits
  33. 04 Apr, 2023 3 commits
    • moto's avatar
      [BC-breaking] Make I/O optional arguments kw-only (#3227) · ab40a3a3
      moto authored 
      Summary:
Recently, we added bunch of options to make StreamReader/Writer flexible. As a result, their methods have many number of arguments, and some of them have semantic grouping.

For example, the arguments of ``StreamWriter.add_video_stream`` are roughly grouped as follow;

- Information about input media format
   `frame_rate`, `width`, `height`, `format`
- Information about encoder
   `encoder`, `encoder_option`
- Information about codec configuration
   `codec_config`
- Information about encode media format
   `encoder_format`, `encoder_frame_rate`, `encoder_width`, `encoder_height`
- Information about additional processing
   `filter_desc`
- Hardware acceleration
   `hw_accel`

We do not know what arguments will be added in the future, but when we do,
we want to keep them roughly grouped, by inserting the new argument
somewhere in a middle without breaking backward compatibility.

This commit puts most of them in keyword-only argument, so that we can
rearrange them without breaking backward compatibility.

Pull Request resolved: https://github.com/pytorch/audio/pull/3227

Reviewed By: hwangjeff

Differential Revision: D44681620

Pulled By: mthrok

fbshipit-source-id: b55f6168f4c2f3d0f59731b9bb0db4ae54e5a90f
      ab40a3a3
    • moto's avatar
      Disable CTC decoder bundle by default (#3232) · 3844a2bd
      moto authored 
      Summary:
As we migrate to use upstream flashlight-text and KenLM, this PR disable building CTC decoder by default.
This will stop shipping flashlight-text and KenLM bundle in torchaudio binary.

Ref: https://github.com/pytorch/audio/issues/3088

cc jacobkahn

Pull Request resolved: https://github.com/pytorch/audio/pull/3232

Reviewed By: hwangjeff

Differential Revision: D44650872

Pulled By: mthrok

fbshipit-source-id: 2415623abaf3cafa181135db5112d3c711137cd7
      3844a2bd
    • hwangjeff's avatar
      Swap in assertions for decoder setup checks (#3235) · ea212c6e
      hwangjeff authored 
      Summary: Pull Request resolved: https://github.com/pytorch/audio/pull/3235

Reviewed By: mthrok

Differential Revision: D44653654

Pulled By: hwangjeff

fbshipit-source-id: f28a6068e826581d76ed4a216adb6019b6486e53
      ea212c6e