Skip to content

GitLab

Commit 6a8a28bb authored by moto's avatar moto Committed by Facebook GitHub Bot
Browse files

Refactor smoke test executions (#2365) 

Summary:
The smoke test jobs simply perform `import torchaudio` to check
if the package artifacts are sane.

Originally, the CI was executing it in the root directory.
This was fine unless the source code is checked out.
When source code is checked out, performing `import torchaudio` in
root directory would import source torchaudio directory, instead of the
installed package.

This error is difficult to notice, so this commit introduces common script to
perform the smoke test, while moving out of root directory.

Pull Request resolved: https://github.com/pytorch/audio/pull/2365

Reviewed By: carolineechen

Differential Revision: D36202069

Pulled By: mthrok

fbshipit-source-id: 4396f85fec5c54869ada4c08f51304539f1b05cf
parent 6beb4875
Hide whitespace changes
Inline Side-by-side
Showing with 45 additions and 16 deletions
.circleci/config.yml
View file @ 6a8a28bb
...@@ -355,11 +355,12 @@ jobs: ...@@ -355,11 +355,12 @@ jobs:
source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION} source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION}
conda install -v -y -c pytorch-${UPLOAD_CHANNEL} pytorch cpuonly conda install -v -y -c pytorch-${UPLOAD_CHANNEL} pytorch cpuonly
conda install -v -y -c file://$HOME/workspace/conda-bld torchaudio conda install -v -y -c file://$HOME/workspace/conda-bld torchaudio
- checkout
- run: - run:
name: smoke test name: smoke test
command: | command: |
source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION} source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION}
python -c "import torchaudio" ./test/smoke_test/run_smoke_test.sh
smoke_test_linux_conda_gpu: smoke_test_linux_conda_gpu:
<<: *smoke_test_common <<: *smoke_test_common
...@@ -375,11 +376,12 @@ jobs: ...@@ -375,11 +376,12 @@ jobs:
source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION} source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION}
conda install -v -y -c pytorch-${UPLOAD_CHANNEL} pytorch cudatoolkit=${CU_VERSION:2:2}.${CU_VERSION:4} -c conda-forge conda install -v -y -c pytorch-${UPLOAD_CHANNEL} pytorch cudatoolkit=${CU_VERSION:2:2}.${CU_VERSION:4} -c conda-forge
conda install -v -y -c file://$HOME/workspace/conda-bld torchaudio conda install -v -y -c file://$HOME/workspace/conda-bld torchaudio
- checkout
- run: - run:
name: smoke test name: smoke test
command: | command: |
source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION} source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION}
python -c "import torchaudio" ./test/smoke_test/run_smoke_test.sh
smoke_test_linux_pip: smoke_test_linux_pip:
<<: *smoke_test_common <<: *smoke_test_common
...@@ -394,11 +396,12 @@ jobs: ...@@ -394,11 +396,12 @@ jobs:
set -x set -x
source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION} source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION}
pip install $(ls ~/workspace/torchaudio*.whl) -f "https://download.pytorch.org/whl/${UPLOAD_CHANNEL}/${CU_VERSION}/torch_${UPLOAD_CHANNEL}.html" pip install $(ls ~/workspace/torchaudio*.whl) -f "https://download.pytorch.org/whl/${UPLOAD_CHANNEL}/${CU_VERSION}/torch_${UPLOAD_CHANNEL}.html"
- checkout
- run: - run:
name: smoke test name: smoke test
command: | command: |
source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION} source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION}
python -c "import torchaudio" ./test/smoke_test/run_smoke_test.sh
smoke_test_windows_conda: smoke_test_windows_conda:
<<: *binary_common <<: *binary_common
...@@ -419,12 +422,13 @@ jobs: ...@@ -419,12 +422,13 @@ jobs:
conda activate python${PYTHON_VERSION} conda activate python${PYTHON_VERSION}
conda install -v -y -c pytorch-${UPLOAD_CHANNEL} pytorch cpuonly conda install -v -y -c pytorch-${UPLOAD_CHANNEL} pytorch cpuonly
conda install -v -y $(ls ~/workspace/conda-bld/win-64/torchaudio*.tar.bz2) conda install -v -y $(ls ~/workspace/conda-bld/win-64/torchaudio*.tar.bz2)
- checkout
- run: - run:
name: smoke test name: smoke test
command: | command: |
eval "$('/C/tools/miniconda3/Scripts/conda.exe' 'shell.bash' 'hook')" eval "$('/C/tools/miniconda3/Scripts/conda.exe' 'shell.bash' 'hook')"
conda activate python${PYTHON_VERSION} conda activate python${PYTHON_VERSION}
python -c "import torchaudio" ./test/smoke_test/run_smoke_test.sh
smoke_test_windows_conda_gpu: smoke_test_windows_conda_gpu:
<<: *binary_common <<: *binary_common
...@@ -460,9 +464,7 @@ jobs: ...@@ -460,9 +464,7 @@ jobs:
conda activate python${PYTHON_VERSION} conda activate python${PYTHON_VERSION}
# Install sound backend # Install sound backend
pip install PySoundFile pip install PySoundFile
# Change to project directory outside of torchaudio root, this is to avoid the installed torchauduio package being shadowed by source directory ./test/smoke_test/run_smoke_test.sh
cd ..
python -c "import torchaudio"
smoke_test_windows_pip: smoke_test_windows_pip:
<<: *binary_common <<: *binary_common
...@@ -482,12 +484,13 @@ jobs: ...@@ -482,12 +484,13 @@ jobs:
conda create -yn python${PYTHON_VERSION} python=${PYTHON_VERSION} conda create -yn python${PYTHON_VERSION} python=${PYTHON_VERSION}
conda activate python${PYTHON_VERSION} conda activate python${PYTHON_VERSION}
pip install $(ls ~/workspace/torchaudio*.whl) -f "https://download.pytorch.org/whl/${UPLOAD_CHANNEL}/torch_${UPLOAD_CHANNEL}.html" pip install $(ls ~/workspace/torchaudio*.whl) -f "https://download.pytorch.org/whl/${UPLOAD_CHANNEL}/torch_${UPLOAD_CHANNEL}.html"
- checkout
- run: - run:
name: smoke test name: smoke test
command: | command: |
eval "$('/C/tools/miniconda3/Scripts/conda.exe' 'shell.bash' 'hook')" eval "$('/C/tools/miniconda3/Scripts/conda.exe' 'shell.bash' 'hook')"
conda activate python${PYTHON_VERSION} conda activate python${PYTHON_VERSION}
python -c "import torchaudio" ./test/smoke_test/run_smoke_test.sh
unittest_linux_cpu: unittest_linux_cpu:
<<: *binary_common <<: *binary_common
......
.circleci/config.yml.in
View file @ 6a8a28bb
...@@ -355,11 +355,12 @@ jobs: ...@@ -355,11 +355,12 @@ jobs:
source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION} source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION}
conda install -v -y -c pytorch-${UPLOAD_CHANNEL} pytorch cpuonly conda install -v -y -c pytorch-${UPLOAD_CHANNEL} pytorch cpuonly
conda install -v -y -c file://$HOME/workspace/conda-bld torchaudio conda install -v -y -c file://$HOME/workspace/conda-bld torchaudio
- checkout
- run: - run:
name: smoke test name: smoke test
command: | command: |
source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION} source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION}
python -c "import torchaudio" ./test/smoke_test/run_smoke_test.sh
smoke_test_linux_conda_gpu: smoke_test_linux_conda_gpu:
<<: *smoke_test_common <<: *smoke_test_common
...@@ -375,11 +376,12 @@ jobs: ...@@ -375,11 +376,12 @@ jobs:
source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION} source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION}
conda install -v -y -c pytorch-${UPLOAD_CHANNEL} pytorch cudatoolkit=${CU_VERSION:2:2}.${CU_VERSION:4} -c conda-forge conda install -v -y -c pytorch-${UPLOAD_CHANNEL} pytorch cudatoolkit=${CU_VERSION:2:2}.${CU_VERSION:4} -c conda-forge
conda install -v -y -c file://$HOME/workspace/conda-bld torchaudio conda install -v -y -c file://$HOME/workspace/conda-bld torchaudio
- checkout
- run: - run:
name: smoke test name: smoke test
command: | command: |
source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION} source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION}
python -c "import torchaudio" ./test/smoke_test/run_smoke_test.sh
smoke_test_linux_pip: smoke_test_linux_pip:
<<: *smoke_test_common <<: *smoke_test_common
...@@ -394,11 +396,12 @@ jobs: ...@@ -394,11 +396,12 @@ jobs:
set -x set -x
source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION} source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION}
pip install $(ls ~/workspace/torchaudio*.whl) -f "https://download.pytorch.org/whl/${UPLOAD_CHANNEL}/${CU_VERSION}/torch_${UPLOAD_CHANNEL}.html" pip install $(ls ~/workspace/torchaudio*.whl) -f "https://download.pytorch.org/whl/${UPLOAD_CHANNEL}/${CU_VERSION}/torch_${UPLOAD_CHANNEL}.html"
- checkout
- run: - run:
name: smoke test name: smoke test
command: | command: |
source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION} source /usr/local/etc/profile.d/conda.sh && conda activate python${PYTHON_VERSION}
python -c "import torchaudio" ./test/smoke_test/run_smoke_test.sh
smoke_test_windows_conda: smoke_test_windows_conda:
<<: *binary_common <<: *binary_common
...@@ -419,12 +422,13 @@ jobs: ...@@ -419,12 +422,13 @@ jobs:
conda activate python${PYTHON_VERSION} conda activate python${PYTHON_VERSION}
conda install -v -y -c pytorch-${UPLOAD_CHANNEL} pytorch cpuonly conda install -v -y -c pytorch-${UPLOAD_CHANNEL} pytorch cpuonly
conda install -v -y $(ls ~/workspace/conda-bld/win-64/torchaudio*.tar.bz2) conda install -v -y $(ls ~/workspace/conda-bld/win-64/torchaudio*.tar.bz2)
- checkout
- run: - run:
name: smoke test name: smoke test
command: | command: |
eval "$('/C/tools/miniconda3/Scripts/conda.exe' 'shell.bash' 'hook')" eval "$('/C/tools/miniconda3/Scripts/conda.exe' 'shell.bash' 'hook')"
conda activate python${PYTHON_VERSION} conda activate python${PYTHON_VERSION}
python -c "import torchaudio" ./test/smoke_test/run_smoke_test.sh
smoke_test_windows_conda_gpu: smoke_test_windows_conda_gpu:
<<: *binary_common <<: *binary_common
...@@ -460,9 +464,7 @@ jobs: ...@@ -460,9 +464,7 @@ jobs:
conda activate python${PYTHON_VERSION} conda activate python${PYTHON_VERSION}
# Install sound backend # Install sound backend
pip install PySoundFile pip install PySoundFile
# Change to project directory outside of torchaudio root, this is to avoid the installed torchauduio package being shadowed by source directory ./test/smoke_test/run_smoke_test.sh
cd ..
python -c "import torchaudio"
smoke_test_windows_pip: smoke_test_windows_pip:
<<: *binary_common <<: *binary_common
...@@ -482,12 +484,13 @@ jobs: ...@@ -482,12 +484,13 @@ jobs:
conda create -yn python${PYTHON_VERSION} python=${PYTHON_VERSION} conda create -yn python${PYTHON_VERSION} python=${PYTHON_VERSION}
conda activate python${PYTHON_VERSION} conda activate python${PYTHON_VERSION}
pip install $(ls ~/workspace/torchaudio*.whl) -f "https://download.pytorch.org/whl/${UPLOAD_CHANNEL}/torch_${UPLOAD_CHANNEL}.html" pip install $(ls ~/workspace/torchaudio*.whl) -f "https://download.pytorch.org/whl/${UPLOAD_CHANNEL}/torch_${UPLOAD_CHANNEL}.html"
- checkout
- run: - run:
name: smoke test name: smoke test
command: | command: |
eval "$('/C/tools/miniconda3/Scripts/conda.exe' 'shell.bash' 'hook')" eval "$('/C/tools/miniconda3/Scripts/conda.exe' 'shell.bash' 'hook')"
conda activate python${PYTHON_VERSION} conda activate python${PYTHON_VERSION}
python -c "import torchaudio" ./test/smoke_test/run_smoke_test.sh
unittest_linux_cpu: unittest_linux_cpu:
<<: *binary_common <<: *binary_common
......
test/smoke_test/run_smoke_test.sh 0 → 100755
View file @ 6a8a28bb
#!/usr/bin/env bash
set -eux
# This script is used by CI to perform smoke tests on installed binaries.
# When `import torchaudio` is executed from the root directory of the repo,
# the source `torchaudio` directory will shadow the actual installation.
# Changing to this directory to avoid that.
cd -- "$( dirname -- "${BASH_SOURCE[0]}" )"
python smoke_test.py
test/smoke_test/smoke_test.py 0 → 100644
View file @ 6a8a28bb
"""Run smoke tests"""
import torchaudio # noqa: F401
import torchaudio.compliance.kaldi # noqa: F401
import torchaudio.datasets # noqa: F401
import torchaudio.functional # noqa: F401
import torchaudio.models # noqa: F401
import torchaudio.pipelines # noqa: F401
import torchaudio.sox_effects # noqa: F401
import torchaudio.transforms # noqa: F401
import torchaudio.utils # noqa: F401
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment