Skip to content

GitLab

"vscode:/vscode.git/clone" did not exist on "2796fbb53df433947b95d7e77a923d7dd8337979"
Commit a9847c38 authored by flyingdown's avatar flyingdown
Browse files

1.修改了readme 

2.添加了dcu_version及dtk的信息
parent e6a3a0e8
Show whitespace changes
Inline Side-by-side
Showing with 236 additions and 221 deletions
README.md
View file @ a9847c38
torchaudio: an audio library for PyTorch
========================================
[![Build Status](https://circleci.com/gh/pytorch/audio.svg?style=svg)](https://app.circleci.com/pipelines/github/pytorch/audio)
[![Documentation](https://img.shields.io/badge/dynamic/json.svg?label=docs&url=https%3A%2F%2Fpypi.org%2Fpypi%2Ftorchaudio%2Fjson&query=%24.info.version&colorB=brightgreen&prefix=v)](https://pytorch.org/audio/)
The aim of torchaudio is to apply [PyTorch](https://github.com/pytorch/pytorch) to
the audio domain. By supporting PyTorch, torchaudio follows the same philosophy
of providing strong GPU acceleration, having a focus on trainable features through
the autograd system, and having consistent style (tensor names and dimension names).
Therefore, it is primarily a machine learning library and not a general signal
processing library. The benefits of PyTorch can be seen in torchaudio through
having all the computations be through PyTorch operations which makes it easy
to use and feel like a natural extension.
- [Support audio I/O (Load files, Save files)](http://pytorch.org/audio/stable/)
- Load a variety of audio formats, such as `wav`, `mp3`, `ogg`, `flac`, `opus`, `sphere`, into a torch Tensor using SoX
- [Kaldi (ark/scp)](http://pytorch.org/audio/stable/kaldi_io.html)
- [Dataloaders for common audio datasets](http://pytorch.org/audio/stable/datasets.html)
- Common audio transforms
- [Spectrogram, AmplitudeToDB, MelScale, MelSpectrogram, MFCC, MuLawEncoding, MuLawDecoding, Resample](http://pytorch.org/audio/stable/transforms.html)
- Compliance interfaces: Run code using PyTorch that align with other libraries
- [Kaldi: spectrogram, fbank, mfcc](https://pytorch.org/audio/stable/compliance.kaldi.html)
Dependencies
------------
* PyTorch (See below for the compatible versions)
* [optional] vesis84/kaldi-io-for-python commit cb46cb1f44318a5d04d4941cf39084c5b021241e or above
The following are the corresponding ``torchaudio`` versions and supported Python versions.
| ``torch`` | ``torchaudio`` | ``python`` |
| ------------------------ | ------------------------ | ------------------------------- |
| ``master`` / ``nightly`` | ``main`` / ``nightly`` | ``>=3.6``, ``<=3.9`` |
| ``1.9.0`` | ``0.9.0`` | ``>=3.6``, ``<=3.9`` |
| ``1.8.0`` | ``0.8.0`` | ``>=3.6``, ``<=3.9`` |
| ``1.7.1`` | ``0.7.2`` | ``>=3.6``, ``<=3.9`` |
| ``1.7.0`` | ``0.7.0`` | ``>=3.6``, ``<=3.8`` |
| ``1.6.0`` | ``0.6.0`` | ``>=3.6``, ``<=3.8`` |
| ``1.5.0`` | ``0.5.0`` | ``>=3.5``, ``<=3.8`` |
| ``1.4.0`` | ``0.4.0`` | ``==2.7``, ``>=3.5``, ``<=3.8`` |
Installation
------------
### Binary Distributions
To install the latest version using anaconda, run:
# TORCHAUDIO
```
conda install -c pytorch torchaudio
```
## 介绍
To install the latest pip wheels, run:
```
pip install torchaudio -f https://download.pytorch.org/whl/torch_stable.html
```
[Introduction](README_ORIGIN.md)
(If you do not have torch already installed, this will default to installing
torch from PyPI. If you need a different torch configuration, preinstall torch
before running this command.)
## 安装
### Nightly build
### System Requirements
Note that nightly build is built on PyTorch's nightly build. Therefore, you need to install the latest PyTorch when you use nightly build of torchaudio.
- Linux.
**pip**
- Python 3.7, 3.8, 3.9
```
pip install --pre torchaudio -f https://download.pytorch.org/whl/nightly/torch_nightly.html
```
- (**推荐**) Upgrade pip
**conda**
```
conda install -y -c pytorch-nightly torchaudio
```
### From Source
On non-Windows platforms, the build process builds libsox and codecs that torchaudio need to link to. It will fetch and build libmad, lame, flac, vorbis, opus, and libsox before building extension. This process requires `cmake` and `pkg-config`. libsox-based features can be disabled with `BUILD_SOX=0`.
The build process also builds the RNN transducer loss. This functionality can be disabled by setting the environment variable `BUILD_RNNT=0`.
```
python3 -m pip install --upgrade pip #--user
```
### 使用pip安装（以0.13.1版本为例）
可以在光合[光合开发者社区](https://developer.hpccube.com/tool/#sdk) AI 生态包中获取最新的 torchaudio Release 版本（需对应 DCU Toolkit 版本与 python 版本）
```bash
# Linux
python setup.py install
# OSX
MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py install
# Windows
# We need to use the MSVC x64 toolset for compilation, with Visual Studio's vcvarsall.bat or directly with vcvars64.bat.
# These batch files are under Visual Studio's installation folder, under 'VC\Auxiliary\Build\'.
# More information available at:
# https://docs.microsoft.com/en-us/cpp/build/how-to-enable-a-64-bit-visual-cpp-toolset-on-the-command-line?view=msvc-160#use-vcvarsallbat-to-set-a-64-bit-hosted-build-architecture
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" x64 && set BUILD_SOX=0 && python setup.py install
# or
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvars64.bat" && set BUILD_SOX=0 && python setup.py install
python3 -m pip install torchaudio-0.13.1+gitd946a7c.abi0.dtk2304-cp37-cp37m-linux_x86_64.whl
```
This is known to work on linux and unix distributions such as Ubuntu and CentOS 7 and macOS.
If you try this on a new system and find a solution to make it work, feel free to share it by opening an issue.
### 使用源码安装
Quick Usage
-----------
#### 编译环境准备（以0.13.1版本为例）
```python
import torchaudio
- 拉取 torchaudio 代码
waveform, sample_rate = torchaudio.load('foo.wav') # load tensor from file
torchaudio.save('foo_save.wav', waveform, sample_rate) # save tensor to file
```
Backend Dispatch
----------------
```
git clone -b 0.13.1-dtk23.04 http://developer.hpccube.com/codes/aicomponent/torchaudio.git
```
By default in OSX and Linux, torchaudio uses SoX as a backend to load and save files.
The backend can be changed to [SoundFile](https://pysoundfile.readthedocs.io/en/latest/)
using the following. See [SoundFile](https://pysoundfile.readthedocs.io/en/latest/)
for installation instructions.
- 在[开发者社区](https://developer.hpccube.com/tool/#sdk) DCU Toolkit 中下载 DTK-23.04 解压至 /opt/ 路径下，并建立软链接
```python
import torchaudio
torchaudio.set_audio_backend("soundfile") # switch backend
```
cd /opt && ln -s dtk-23.04 dtk
```
waveform, sample_rate = torchaudio.load('foo.wav') # load tensor from file, as usual
torchaudio.save('foo_save.wav', waveform, sample_rate) # save tensor to file, as usual
- 在光合[光合开发者社区](https://developer.hpccube.com/tool/#sdk) AI 生态包中获取对应的 pytorch Release 版本（需对应 DCU Toolkit 版本与 python 版本）
```bash
python3 -m pip install torch-1.13.1a0+git4c8a1fe.abi0.dtk2304-cp37-cp37m-linux_x86_64.whl
```
**Note**
- SoundFile currently does not support mp3.
- "soundfile" backend is not supported by TorchScript.
- 导入环境变量以及安装必要依赖库
API Reference
-------------
```bash
source /opt/dtk/env.sh
API Reference is located here: http://pytorch.org/audio/
export PYTORCH_ROCM_ARCH="gfx906;gfx926"
Contributing Guidelines
-----------------------
MAX_JOBS=16
pip3 install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple --trusted-host pypi.tuna.tsinghua.edu.cn
```
Please refer to [CONTRIBUTING.md](./CONTRIBUTING.md)
Disclaimer on Datasets
----------------------
#### 编译安装
This is a utility library that downloads and prepares public datasets. We do not host or distribute these datasets, vouch for their quality or fairness, or claim that you have license to use the dataset. It is your responsibility to determine whether you have permission to use the dataset under the dataset's license.
- 执行编译命令
```shell
cd audio
CXX=hipcc CC=hipcc python3 setup.py bdist_wheel
pip install dist/torchaudio*
```
If you're a dataset owner and wish to update any part of it (description, citation, etc.), or do not want your dataset to be included in this library, please get in touch through a GitHub issue. Thanks for your contribution to the ML community!
README_HIP.md deleted 100644 → 0
View file @ e6a3a0e8
# TORCHAUDIO
## 安装
### System Requirements
- Linux.
- Python 3.7, 3.8, 3.9
- (**推荐**) Upgrade pip
```
python3 -m pip install --upgrade pip #--user
```
### 使用pip安装（以0.13.1版本为例）
可以在光合[光合开发者社区](https://developer.hpccube.com/tool/#sdk) AI 生态包中获取最新的 torchaudio Release 版本（需对应 DCU Toolkit 版本与 python 版本）
```bash
python3 -m pip install torchaudio-0.13.1+gitd946a7c.abi0.dtk2304-cp37-cp37m-linux_x86_64.whl
```
### 使用源码安装
#### 编译环境准备（以0.13.1版本为例）
- 拉取 torchaudio 代码
```
git clone -b 0.13.1-dtk23.04 http://developer.hpccube.com/codes/aicomponent/torchaudio.git
```
- 在[开发者社区](https://developer.hpccube.com/tool/#sdk) DCU Toolkit 中下载 DTK-23.04 解压至 /opt/ 路径下，并建立软链接
```
cd /opt && ln -s dtk-23.04 dtk
```
- 在光合[光合开发者社区](https://developer.hpccube.com/tool/#sdk) AI 生态包中获取对应的 pytorch Release 版本（需对应 DCU Toolkit 版本与 python 版本）
```bash
python3 -m pip install torch-1.13.1a0+git4c8a1fe.abi0.dtk2304-cp37-cp37m-linux_x86_64.whl
```
- 导入环境变量以及安装必要依赖库
```bash
source /opt/dtk/env.sh
export PYTORCH_ROCM_ARCH="gfx906;gfx926"
MAX_JOBS=16
sha=`git rev-parse HEAD`
export BUILD_VERSION=0.13.1+git${sha:0:7}.abi0.dtk2304
pip3 install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple --trusted-host pypi.tuna.tsinghua.edu.cn
```
#### 编译安装
- 执行编译命令
```shell
cd audio
CXX=hipcc CC=hipcc python3 setup.py bdist_wheel
pip install dist/torchaudio*
```
README_ORIGIN.md 0 → 100644
View file @ a9847c38
torchaudio: an audio library for PyTorch
========================================
[![Build Status](https://circleci.com/gh/pytorch/audio.svg?style=svg)](https://app.circleci.com/pipelines/github/pytorch/audio)
[![Documentation](https://img.shields.io/badge/dynamic/json.svg?label=docs&url=https%3A%2F%2Fpypi.org%2Fpypi%2Ftorchaudio%2Fjson&query=%24.info.version&colorB=brightgreen&prefix=v)](https://pytorch.org/audio/)
The aim of torchaudio is to apply [PyTorch](https://github.com/pytorch/pytorch) to
the audio domain. By supporting PyTorch, torchaudio follows the same philosophy
of providing strong GPU acceleration, having a focus on trainable features through
the autograd system, and having consistent style (tensor names and dimension names).
Therefore, it is primarily a machine learning library and not a general signal
processing library. The benefits of PyTorch can be seen in torchaudio through
having all the computations be through PyTorch operations which makes it easy
to use and feel like a natural extension.
- [Support audio I/O (Load files, Save files)](http://pytorch.org/audio/stable/)
- Load a variety of audio formats, such as `wav`, `mp3`, `ogg`, `flac`, `opus`, `sphere`, into a torch Tensor using SoX
- [Kaldi (ark/scp)](http://pytorch.org/audio/stable/kaldi_io.html)
- [Dataloaders for common audio datasets](http://pytorch.org/audio/stable/datasets.html)
- Common audio transforms
- [Spectrogram, AmplitudeToDB, MelScale, MelSpectrogram, MFCC, MuLawEncoding, MuLawDecoding, Resample](http://pytorch.org/audio/stable/transforms.html)
- Compliance interfaces: Run code using PyTorch that align with other libraries
- [Kaldi: spectrogram, fbank, mfcc](https://pytorch.org/audio/stable/compliance.kaldi.html)
Dependencies
------------
* PyTorch (See below for the compatible versions)
* [optional] vesis84/kaldi-io-for-python commit cb46cb1f44318a5d04d4941cf39084c5b021241e or above
The following are the corresponding ``torchaudio`` versions and supported Python versions.
| ``torch`` | ``torchaudio`` | ``python`` |
| ------------------------ | ------------------------ | ------------------------------- |
| ``master`` / ``nightly`` | ``main`` / ``nightly`` | ``>=3.6``, ``<=3.9`` |
| ``1.9.0`` | ``0.9.0`` | ``>=3.6``, ``<=3.9`` |
| ``1.8.0`` | ``0.8.0`` | ``>=3.6``, ``<=3.9`` |
| ``1.7.1`` | ``0.7.2`` | ``>=3.6``, ``<=3.9`` |
| ``1.7.0`` | ``0.7.0`` | ``>=3.6``, ``<=3.8`` |
| ``1.6.0`` | ``0.6.0`` | ``>=3.6``, ``<=3.8`` |
| ``1.5.0`` | ``0.5.0`` | ``>=3.5``, ``<=3.8`` |
| ``1.4.0`` | ``0.4.0`` | ``==2.7``, ``>=3.5``, ``<=3.8`` |
Installation
------------
### Binary Distributions
To install the latest version using anaconda, run:
```
conda install -c pytorch torchaudio
```
To install the latest pip wheels, run:
```
pip install torchaudio -f https://download.pytorch.org/whl/torch_stable.html
```
(If you do not have torch already installed, this will default to installing
torch from PyPI. If you need a different torch configuration, preinstall torch
before running this command.)
### Nightly build
Note that nightly build is built on PyTorch's nightly build. Therefore, you need to install the latest PyTorch when you use nightly build of torchaudio.
**pip**
```
pip install --pre torchaudio -f https://download.pytorch.org/whl/nightly/torch_nightly.html
```
**conda**
```
conda install -y -c pytorch-nightly torchaudio
```
### From Source
On non-Windows platforms, the build process builds libsox and codecs that torchaudio need to link to. It will fetch and build libmad, lame, flac, vorbis, opus, and libsox before building extension. This process requires `cmake` and `pkg-config`. libsox-based features can be disabled with `BUILD_SOX=0`.
The build process also builds the RNN transducer loss. This functionality can be disabled by setting the environment variable `BUILD_RNNT=0`.
```bash
# Linux
python setup.py install
# OSX
MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py install
# Windows
# We need to use the MSVC x64 toolset for compilation, with Visual Studio's vcvarsall.bat or directly with vcvars64.bat.
# These batch files are under Visual Studio's installation folder, under 'VC\Auxiliary\Build\'.
# More information available at:
# https://docs.microsoft.com/en-us/cpp/build/how-to-enable-a-64-bit-visual-cpp-toolset-on-the-command-line?view=msvc-160#use-vcvarsallbat-to-set-a-64-bit-hosted-build-architecture
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" x64 && set BUILD_SOX=0 && python setup.py install
# or
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvars64.bat" && set BUILD_SOX=0 && python setup.py install
```
This is known to work on linux and unix distributions such as Ubuntu and CentOS 7 and macOS.
If you try this on a new system and find a solution to make it work, feel free to share it by opening an issue.
Quick Usage
-----------
```python
import torchaudio
waveform, sample_rate = torchaudio.load('foo.wav') # load tensor from file
torchaudio.save('foo_save.wav', waveform, sample_rate) # save tensor to file
```
Backend Dispatch
----------------
By default in OSX and Linux, torchaudio uses SoX as a backend to load and save files.
The backend can be changed to [SoundFile](https://pysoundfile.readthedocs.io/en/latest/)
using the following. See [SoundFile](https://pysoundfile.readthedocs.io/en/latest/)
for installation instructions.
```python
import torchaudio
torchaudio.set_audio_backend("soundfile") # switch backend
waveform, sample_rate = torchaudio.load('foo.wav') # load tensor from file, as usual
torchaudio.save('foo_save.wav', waveform, sample_rate) # save tensor to file, as usual
```
**Note**
- SoundFile currently does not support mp3.
- "soundfile" backend is not supported by TorchScript.
API Reference
-------------
API Reference is located here: http://pytorch.org/audio/
Contributing Guidelines
-----------------------
Please refer to [CONTRIBUTING.md](./CONTRIBUTING.md)
Disclaimer on Datasets
----------------------
This is a utility library that downloads and prepares public datasets. We do not host or distribute these datasets, vouch for their quality or fairness, or claim that you have license to use the dataset. It is your responsibility to determine whether you have permission to use the dataset under the dataset's license.
If you're a dataset owner and wish to update any part of it (description, citation, etc.), or do not want your dataset to be included in this library, please get in touch through a GitHub issue. Thanks for your contribution to the ML community!
setup.py
View file @ a9847c38
......@@ -7,14 +7,17 @@ from pathlib import Path
from setuptools import setup, find_packages
import distutils.command.clean
import torch
from torch.utils.cpp_extension import ROCM_HOME
from build_tools import setup_helpers
ROOT_DIR = Path(__file__).parent.resolve()
def _run_cmd(cmd, default):
def _run_cmd(cmd, default, shell=False):
try:
return subprocess.check_output(cmd, cwd=ROOT_DIR).decode('ascii').strip()
return subprocess.check_output(cmd, cwd=ROOT_DIR, shell=shell).decode('ascii').strip()
except Exception:
return default
......@@ -29,10 +32,19 @@ elif sha != 'Unknown':
version += '+' + sha[:7]
print('-- Building version ' + version)
abi = _run_cmd(["echo '#include <string>' | gcc -x c++ -E -dM - | fgrep _GLIBCXX_USE_CXX11_ABI | awk '{print $3}'"], '0', shell=True)
dtk = _run_cmd(["cat", os.path.join(ROCM_HOME, '.info/rocm_version')], '0.0.0')
dtk = ''.join(dtk.split('.')[:2])
torch_version = torch.__version__
dcu_version = f"{version}.abi{abi}.dtk{dtk}.torch{torch_version}"
version_path = ROOT_DIR / 'torchaudio' / 'version.py'
with open(version_path, 'w') as f:
f.write("__version__ = '{}'\n".format(version))
f.write("git_version = {}\n".format(repr(sha)))
f.write(f"abi = 'abi{abi}'\n")
f.write(f"dtk = '{dtk}'\n")
f.write(f"torch_version = '{torch_version}'\n")
f.write(f"dcu_version = '{dcu_version}'\n")
pytorch_package_version = os.getenv('PYTORCH_VERSION')
......@@ -86,7 +98,7 @@ def _get_packages():
setup(
name="torchaudio",
version=version,
version=dcu_version,
description="An audio package for PyTorch",
url="https://github.com/pytorch/audio",
author="Soumith Chintala, David Pollack, Sean Naren, Peter Goldsborough",
......
torchaudio/__init__.py
View file @ a9847c38
......@@ -18,7 +18,8 @@ from torchaudio.backend import (
)
try:
from .version import __version__, git_version # noqa: F401
from .version import __version__, git_version, abi, dtk, torch_version, dcu_version # noqa: F401
__dcu_version__ = dcu_version
except ImportError:
pass
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment