Skip to content

GitLab

Commit cd4ab535 authored by Khalique Ahmed's avatar Khalique Ahmed
Browse files

manual merge

parents 3891ee58 a0fa3742
Show whitespace changes
Inline Side-by-side
Showing with 774 additions and 324 deletions
.dockerignore 0 → 100644
View file @ cd4ab535
# Ignore everything
**
# Allow files and directories
!*.txt
!*.ini
!/tools/*.sh
!/test/onnx/.onnxrt-commit
!/docs/.sphinx/*.txt
.github/dependabot.yml 0 → 100644
View file @ cd4ab535
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
version: 2
updates:
- package-ecosystem: "pip" # See documentation for possible values
directory: "/docs/.sphinx" # Location of package manifests
schedule:
interval: "daily"
- package-ecosystem: "pip"
directory: "/tools/accuracy"
schedule:
interval: "daily"
- package-ecosystem: "pip"
directory: "/examples/vision/python_unet"
schedule:
interval: "daily"
- package-ecosystem: "pip"
directory: "/examples/vision/python_super_resolution"
schedule:
interval: "daily"
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "daily"
.github/workflows/README.md 0 → 100644
View file @ cd4ab535
# Workflows
## `add-to-project.yaml`
<p>
This workflow adds pull requests and issues to a specific GitHub project board when they are opened.
</p>
- ## Trigger
The workflow is triggered by the following events:
- A pull request being opened.
- An issue being opened.
- ## Jobs
The workflow has a single job named `add-to-project`. The following step is executed in this job:
- The `add-to-project` job uses the `actions/add-to-project@v0.4.0` action to add pull requests and issues to a specific project board. The `with` parameters are `project-url` and `github-token`, which specify the URL of the project board and the GitHub token used to authenticate the action.
For more details, please refer to the [add-to-project.yaml](https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/blob/develop/.github/workflows/add-to-project.yaml) file in the repository.
---
## `benchmark.yaml`
<p>
This workflow runs the `MiGraphX performance benchmarks` and generates reports by comparing the results with the reference data.
</p>
- ## Trigger
TODO: Update [benchmarks.yml (archived)](https://github.com/ROCmSoftwarePlatform/actions/blob/main/.github/workflows/benchmarks.yml) link after workflow is updated
- The workflow is triggered manually through the "Run workflow" button in the Actions tab of the repository and it will run reusable workflow [benchmarks.yml (archived)](https://github.com/ROCmSoftwarePlatform/actions/blob/main/.github/workflows/benchmarks.yml)
- ## Input Parameters
The workflow uses the following input parameters:
- `rocm_version`: the version of ROCm to use for running the benchmarks.
- `script_repo`: repository that contains the benchmark scripts.
- `result_path`: the path where benchmark results will be stored.
- `result_repo`: the repository where the benchmark results will be pushed for comparison.
For more details, please refer to the [benchmark.yaml](https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/blob/develop/.github/workflows/benchmark.yaml) file in the repository.
---
## `ci.yaml`
<p>
Overall, this workflow automates the process of building and testing the AMDMIGraphX project across multiple platforms and versions.
</p>
- ## Trigger
The workflow is triggered by the following events:
- A pull request being opened, synchronized or closed.
- On push to the `develop`, `master`, and `release/**` branches.
- ## Jobs
The following jobs are executed in the workflow:
- `cancel`: This job is responsible for canceling any previous runs of the workflow that may still be running. It runs on an `ubuntu-latest` runner and uses the `styfle/cancel-workflow-action` action to cancel any previous runs of the workflow.
- `tidy`: It runs on an `ubuntu-20.04` runner and runs `clang-tidy` for the codebase in a Docker container with the MIGraphX build environment.
- `cppcheck`: It runs on an `ubuntu-20.04` runner and performs static analysis on code in a Docker container, and caches the results for faster subsequent runs.
- `format`: It runs on an `ubuntu-20.04` runner and includes steps for freeing up disk space, caching Docker layers, and checking code formatting.
- `pyflakes`: It runs on an `ubuntu-20.04` runner and runs the Pyflakes static analysis tool to detect and report Python code issues.
- `licensing`: It runs on an `ubuntu-20.04` runner and includes steps to free up space, checkout the code, set up Python and run a license check using a Python script.
We have 2 jobs with multiple matrix configurations, both of them are running on `ubuntu-20.04` runner but right now only `linux` works on all 3 configurations (debug, release, codecov) ,`linux-fpga` works just on (debug).
- `linux`: this job runs continuous integration tests for AMDMIGraphX on a Linux operating system. It tests a variety of build configurations to ensure code quality and compatibility.
- `linux-fpga`: this job builds and tests AMDMIGraphX on a Linux operating system with support for FPGA acceleration. It includes additional steps to verify FPGA functionality and performance.
For more details, please refer to the [ci.yaml](https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/blob/develop/.github/workflows/ci.yaml) file in the repository.
---
## `clean-closed-pr-caches.yaml`
<p>
This workflow has purpose to clean up any cached data related to the pull request.
</p>
- ## Trigger
The workflow is triggered by the following events:
- A pull request being closed.
- ## Jobs
The workflow has a single job named `cleanup`. The following steps are executed in this job:
- `Check out code`: step checks out the codebase from the repository.
- `Cleanup`: step performs the actual cache cleanup using a series of commands.
For more details, please refer to the [clean-closed-pr-caches.yaml](https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/blob/develop/.github/workflows/clean-closed-pr-caches.yaml) file in the repository.
---
## `history.yaml`
<p>
This workflow generates a report of the MiGraphX benchmark results between two dates and sends it to a specified email address. The report is also uploaded to a specified repository.
</p>
- ## Trigger
- The workflow is triggered manually through the "Run workflow" button in the Actions tab of the repository and it will run reusable workflow [history.yml](https://github.com/ROCmSoftwarePlatform/migraphx-benchmark/blob/main/.github/workflows/history.yml)
- ## Input Parameters
The workflow requires the following inputs:
- `start_date`: Start date for results analysis.
- `end_date`: End date for results analysis.
- `history_repo`: Repository for history results between dates.
- `benchmark_utils_repo`: Repository where benchmark utils are stored.
- `organization`: Organization based on which location of files will be different.
For more details, please refer to the [history.yaml](https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/blob/develop/.github/workflows/history.yaml) file in the repository.
---
## `performance.yaml`
<p>
This workflow runs performance tests on the MIGraphX repository and generates a report of the results.
</p>
- ## Trigger
The workflow will run reusable workflow [perf-test.yml](https://github.com/ROCmSoftwarePlatform/migraphx-benchmark/blob/main/.github/workflows/perf-test.yml) by the following events:
- Pull requests opened, synchronized or closed on the `develop` branch.
- Schedule: Runs every day of the week from Monday to Saturday at 6:00 AM.
- Manual trigger through the "Run workflow" button in the Actions tab of the repository.
- ## Input Parameters
The workflow requires the following inputs:
- `rocm_release`: ROCm version to use for the performance tests.
- `performance_reports_repo`: Repository where the performance reports are stored.
- `benchmark_utils_repo`: Repository where the benchmark utilities are stored.
- `organization`: Organization based on which location of files will be different.
- `result_number`: Last N results.
- `model_timeout`: If a model in the performance test script passes this threshold, it will be skipped.
- `flags`: Command line arguments to be passed to the performance test script. Default is `-r`.
For more details, please refer to the [performance.yaml](https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/blob/develop/.github/workflows/performance.yaml) file in the repository.
---
## `rocm-image-release.yaml`
<p>
This workflow builds a Docker image for a specified ROCm release version and pushes it to the specified repository. If image already exists nothing will happen, and there is also option to overwrite existing image.
</p>
- ## Trigger
- The workflow is triggered manually through the "Run workflow" button in the Actions tab of the repository and it will run reusable workflow [rocm-release.yml](https://github.com/ROCmSoftwarePlatform/migraphx-benchmark/blob/main/.github/workflows/rocm-release.yml)
- ## Input Parameters
The workflow requires the following inputs:
- `rocm_release`: ROCm release version to build Docker image for.
- `benchmark_utils_repo`: Repository where benchmark utils are stored.
- `base_image`: Base image for ROCm Docker build.
- `docker_image`: Docker image name for ROCm Docker build.
- `build_navi`: Build number for the Navi architecture.
- `organization`: The organization name used to determine the location of files.
- `overwrite`: Specify whether to overwrite the Docker image if it already exists.
For more details, please refer to the [rocm-image-release.yaml](https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/blob/develop/.github/workflows/rocm-image-release.yaml) file in the repository.
---
## `sync-onnxrt-main.yaml`
<p>
This workflow updates a file with the latest commit hash then creates a pull request using the updated commit hash and adds labels, assignees, reviewers, and a title and body to describe the changes.
</p>
- ## Trigger
The workflow is triggered by the following events:
- Schedule: Runs every week on Friday at 05:07 PM.
- ## Jobs
The workflow has a single job named `Update and create pull request`. The following steps are executed in this job:
- `get_date`: step sets an environment variable to the current date in the format 'YYYY-MM-DD'.
- `extract_sha1`: step fetches the latest SHA1 commit hash of the HEAD branch of the `microsoft/onnxruntime` repository and sets it as an environment variable.
- `echo_sha1`: step prints the SHA1 commit hash set in step `extract_sha1`.
- `actions/checkout@v3`: step checks out the codebase from the repository.
- `update_file`: step updates a file in the repository with the SHA1 commit hash fetched in step `extract_sha1`.
- `Make changes to pull request`: step uses the `peter-evans/create-pull-request` action to create a pull request.
For more details, please refer to the [sync-onnxrt-main.yaml](https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/blob/develop/.github/workflows/sync-onnxrt-main.yaml) file in the repository.
---
.github/workflows/ci.yaml
View file @ cd4ab535
name: migraphx name: migraphx
on: [push, pull_request] on:
pull_request:
push:
branches:
- develop
- master
- 'release/**'
jobs: jobs:
cancel: cancel:
...@@ -15,44 +22,40 @@ jobs: ...@@ -15,44 +22,40 @@ jobs:
steps: steps:
- name: Free space - name: Free space
run: | uses: jlumbroso/free-disk-space@main
sudo rm -rf /usr/local/android /usr/share/dotnet /usr/local/share/boost /opt/ghc /usr/local/share/chrom* /usr/share/swift /usr/local/julia* /usr/local/lib/android /usr/local/graalvm /usr/local/aws* /usr/local/lib/heroku with:
du . --max-depth=1 -h tool-cache: true
ls -la android: true
cd /usr/local dotnet: true
du . --max-depth=1 -h haskell: true
ls -la large-packages: true
cd /usr/local/lib swap-storage: true
echo $(pwd)
du . --max-depth=1 -h
ls -la
- uses: actions/checkout@v3 - uses: actions/checkout@v3
# In this step, this action saves a list of existing images, # In this step, this action saves a list of existing images,
# the cache is created without them in the post run. # the cache is created without them in the post run.
# It also restores the cache if it exists. # It also restores the cache if it exists.
# name: Docker Layer Caching2 - name: Docker layer cache
- uses: jpribyl/action-docker-layer-caching@v0.1.1 uses: jpribyl/action-docker-layer-caching@v0.1.1
with:
key: docker-layer-caching-migraphx-${{hashFiles('hip-clang.docker', '**/*requirements.txt', '**/install_prereqs.sh', 'rbuild.ini')}}
restore-keys:
docker-layer-caching-migraphx-
# Ignore the failure of a step and avoid terminating the job. # Ignore the failure of a step and avoid terminating the job.
continue-on-error: true continue-on-error: true
- name: Prepare timestamp - name: Restore cache files for tidy
id: cache_timestamp uses: actions/cache/restore@v3
shell: bash id: tidy_restore
run: echo timestamp="$(date +'%Y-%m-%dT%H:%M:%S')" >> $GITHUB_OUTPUT
- name: Cache files for tidy
uses: pat-s/always-upload-cache@v3.0.11
with: with:
path: tidy-cache path: tidy-cache
key: tidy-cache-${{ steps.cache_timestamp.outputs.timestamp }} key: tidy-cache-${{ github.ref }}
restore-keys: | restore-keys: tidy-cache-
tidy-cache-${{ steps.cache_timestamp.outputs.timestamp }}
tidy-cache-
- name: Build the Docker image - name: Build the Docker image
run: docker build . --file hip-clang.docker --tag migraphx run: |
docker build . --file hip-clang.docker --tag migraphx
- name: Clang tidy - name: Clang tidy
shell: bash -c "docker run -i -v=$GITHUB_WORKSPACE:/data -w /data migraphx bash < {0}" shell: bash -c "docker run -i -v=$GITHUB_WORKSPACE:/data -w /data migraphx bash < {0}"
...@@ -70,34 +73,61 @@ jobs: ...@@ -70,34 +73,61 @@ jobs:
.. ..
make -j2 -k onnx-proto tf-proto tidy make -j2 -k onnx-proto tf-proto tidy
# GH actions can not update existing cache, as a workaround clear cache and then save it
- name: Clear tidy cache before saving
if: ${{ steps.tidy_restore.outputs.cache-hit }}
shell: bash
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
gh extension install actions/gh-actions-cache --pin v1.0.1
gh actions-cache delete ${{ steps.tidy_restore.outputs.cache-matched-key }} --confirm
continue-on-error: true
- name: Save cache files for tidy
uses: actions/cache/save@v3
if: always()
with:
path: tidy-cache
key: tidy-cache-${{ github.ref }}
cppcheck: cppcheck:
runs-on: ubuntu-20.04 runs-on: ubuntu-20.04
steps: steps:
- name: Free space - name: Free space
run: sudo rm -rf /usr/local/android /usr/share/dotnet /usr/local/share/boost /opt/ghc /usr/local/share/chrom* /usr/share/swift /usr/local/julia* /usr/local/lib/android /usr/local/graalvm /usr/local/aws* /usr/local/lib/heroku uses: jlumbroso/free-disk-space@main
with:
tool-cache: true
android: true
dotnet: true
haskell: true
large-packages: true
swap-storage: true
- uses: actions/checkout@v3 - uses: actions/checkout@v3
# In this step, this action saves a list of existing images, # In this step, this action saves a list of existing images,
# the cache is created without them in the post run. # the cache is created without them in the post run.
# It also restores the cache if it exists. # It also restores the cache if it exists.
- uses: jpribyl/action-docker-layer-caching@v0.1.1 - name: Docker layer cache
uses: jpribyl/action-docker-layer-caching@v0.1.1
with:
key: docker-layer-caching-migraphx-${{hashFiles('hip-clang.docker', '**/*requirements.txt', '**/install_prereqs.sh', 'rbuild.ini')}}
restore-keys:
docker-layer-caching-migraphx-
# Ignore the failure of a step and avoid terminating the job. # Ignore the failure of a step and avoid terminating the job.
continue-on-error: true continue-on-error: true
- name: Prepare timestamp - name: Restore cache files for cppcheck
id: cache_timestamp id: cppcheck_restore
shell: bash uses: actions/cache/restore@v3
run: echo timestamp="$(date +'%Y-%m-%dT%H:%M:%S')" >> $GITHUB_OUTPUT
- name: Cache files for cppcheck
uses: pat-s/always-upload-cache@v2.1.3
with: with:
path: cppcheck-cache path: cppcheck-cache
key: cppcheck-cache-${{ hashFiles('cppcheck.rules', 'CMakeLists.txt') }}-${{ steps.cache_timestamp.outputs.timestamp }} key: cppcheck-cache-${{ hashFiles('cppcheck.rules', 'CMakeLists.txt') }}-${{ github.ref }}
restore-keys: | restore-keys: cppcheck-cache-${{ hashFiles('cppcheck.rules', 'CMakeLists.txt') }}-
cppcheck-cache-${{ hashFiles('cppcheck.rules', 'CMakeLists.txt') }}-${{ steps.cache_timestamp.outputs.timestamp }}
cppcheck-cache-${{ hashFiles('cppcheck.rules', 'CMakeLists.txt') }}-
- name: Build the Docker image - name: Build the Docker image
run: docker build . --file hip-clang.docker --tag migraphx run: docker build . --file hip-clang.docker --tag migraphx
...@@ -114,18 +144,49 @@ jobs: ...@@ -114,18 +144,49 @@ jobs:
.. ..
make -j2 cppcheck make -j2 cppcheck
# GH actions can not update existing cache, as a workaround clear cache and then save it
- name: Clear cppcheck cache before saving
if: ${{ steps.cppcheck_restore.outputs.cache-hit }}
shell: bash
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
gh extension install actions/gh-actions-cache --pin v1.0.1
gh actions-cache delete ${{ steps.cppcheck_restore.outputs.cache-matched-key }} --confirm
continue-on-error: true
- name: Save cache files for cppcheck
uses: actions/cache/save@v3
if: always()
with:
path: cppcheck-cache
key: cppcheck-cache-${{ hashFiles('cppcheck.rules', 'CMakeLists.txt') }}-${{ github.ref }}
format: format:
runs-on: ubuntu-20.04 runs-on: ubuntu-20.04
steps: steps:
- name: Free space - name: Free space
run: sudo rm -rf /usr/local/android /usr/share/dotnet /usr/local/share/boost /opt/ghc /usr/local/share/chrom* /usr/share/swift /usr/local/julia* /usr/local/lib/android /usr/local/graalvm /usr/local/aws* /usr/local/lib/heroku uses: jlumbroso/free-disk-space@main
with:
tool-cache: true
android: true
dotnet: true
haskell: true
large-packages: true
swap-storage: true
- uses: actions/checkout@v3 - uses: actions/checkout@v3
# In this step, this action saves a list of existing images, # In this step, this action saves a list of existing images,
# the cache is created without them in the post run. # the cache is created without them in the post run.
# It also restores the cache if it exists. # It also restores the cache if it exists.
- uses: jpribyl/action-docker-layer-caching@v0.1.1 - name: Docker layer cache
uses: jpribyl/action-docker-layer-caching@v0.1.1
with:
key: docker-layer-caching-migraphx-${{hashFiles('hip-clang.docker', '**/*requirements.txt', '**/install_prereqs.sh', 'rbuild.ini')}}
restore-keys:
docker-layer-caching-migraphx-
# Ignore the failure of a step and avoid terminating the job. # Ignore the failure of a step and avoid terminating the job.
continue-on-error: true continue-on-error: true
...@@ -155,7 +216,15 @@ jobs: ...@@ -155,7 +216,15 @@ jobs:
steps: steps:
- name: Free space - name: Free space
run: sudo rm -rf /usr/local/android /usr/share/dotnet /usr/local/share/boost /opt/ghc /usr/local/share/chrom* /usr/share/swift /usr/local/julia* /usr/local/lib/android /usr/local/graalvm /usr/local/aws* /usr/local/lib/heroku uses: jlumbroso/free-disk-space@main
with:
tool-cache: true
android: true
dotnet: true
haskell: true
large-packages: true
swap-storage: true
- uses: actions/checkout@v3 - uses: actions/checkout@v3
- name: Set up Python - name: Set up Python
uses: actions/setup-python@v4 uses: actions/setup-python@v4
...@@ -167,7 +236,7 @@ jobs: ...@@ -167,7 +236,7 @@ jobs:
- name: Run pyflakes - name: Run pyflakes
run: | run: |
pyflakes --version pyflakes --version
pyflakes examples/ tools/ src/ test/ doc/ pyflakes examples/ tools/ src/ test/ docs/
mypy --version mypy --version
mypy tools/api.py mypy tools/api.py
...@@ -176,7 +245,15 @@ jobs: ...@@ -176,7 +245,15 @@ jobs:
steps: steps:
- name: Free space - name: Free space
run: sudo rm -rf /usr/local/android /usr/share/dotnet /usr/local/share/boost /opt/ghc /usr/local/share/chrom* /usr/share/swift /usr/local/julia* /usr/local/lib/android /usr/local/graalvm /usr/local/aws* /usr/local/lib/heroku uses: jlumbroso/free-disk-space@main
with:
tool-cache: true
android: true
dotnet: true
haskell: true
large-packages: true
swap-storage: true
- uses: actions/checkout@v3 - uses: actions/checkout@v3
- name: Set up Python - name: Set up Python
uses: actions/setup-python@v4 uses: actions/setup-python@v4
...@@ -207,7 +284,21 @@ jobs: ...@@ -207,7 +284,21 @@ jobs:
steps: steps:
- name: Free space - name: Free space
run: sudo rm -rf /usr/local/android /usr/share/dotnet /usr/local/share/boost /opt/ghc /usr/local/share/chrom* /usr/share/swift /usr/local/julia* /usr/local/lib/android /usr/local/graalvm /usr/local/aws* /usr/local/lib/heroku uses: jlumbroso/free-disk-space@main
with:
tool-cache: true
android: true
dotnet: true
haskell: true
large-packages: true
swap-storage: true
- name : Install rbuild and lld
run: |
sudo apt-get install -y lld
python -m pip install --upgrade pip
pip install https://github.com/RadeonOpenCompute/rbuild/archive/master.tar.gz
- uses: actions/checkout@v3 - uses: actions/checkout@v3
- name: Set up Python - name: Set up Python
uses: actions/setup-python@v4 uses: actions/setup-python@v4
...@@ -217,36 +308,25 @@ jobs: ...@@ -217,36 +308,25 @@ jobs:
# Ignore the failure of a step and avoid terminating the job. # Ignore the failure of a step and avoid terminating the job.
continue-on-error: true continue-on-error: true
uses: actions/cache@v3 uses: actions/cache@v3
id: deps_cache
with: with:
# This path is specific to Ubuntu # This path is specific to Ubuntu
path: ${{ github.workspace }}/cget path: ${{ github.workspace }}/cget
# Look to see if there is a cache hit for the corresponding requirements file # Look to see if there is a cache hit for the corresponding requirements file
key: key: ${{ matrix.os }}-cget-4-${{ hashFiles('requirements.txt', 'dev-requirements.txt') }}
${{ matrix.os }}-cget-4-${{ hashFiles('requirements.txt', 'dev-requirements.txt') }} restore-keys: ${{ matrix.os }}-cget-4-
${{ matrix.os }}-cget-4-
- name: Install dependencies - name: Install dependencies
run: | if: steps.deps_cache.outputs.cache-hit != 'true'
python -m pip install --upgrade pip run: rbuild prepare -d cget -s gh
pip install https://github.com/RadeonOpenCompute/rbuild/archive/master.tar.gz
rbuild prepare -d cget -s gh
sudo apt-get install -y lld
- name: Prepare timestamp
id: cache_timestamp
shell: bash
run: echo timestamp="$(date +'%Y-%m-%dT%H:%M:%S')" >> $GITHUB_OUTPUT
- name: Cache files for ccache - name: Restore cache files for ccache
# Ignore the failure of a step and avoid terminating the job. uses: actions/cache/restore@v3
continue-on-error: true id: ccache_restore
uses: pat-s/always-upload-cache@v2.1.3
with: with:
path: ccache path: ${{ github.workspace }}/ccache
key: ${{ matrix.os }}-${{ matrix.configuration }}-ccache-${{ steps.cache_timestamp.outputs.timestamp }} key: ${{ matrix.os }}-${{ matrix.configuration }}-ccache-${{ github.ref }}
restore-keys: | restore-keys: ${{ matrix.os }}-${{ matrix.configuration }}-ccache-
${{ matrix.os }}-${{ matrix.configuration }}-ccache-${{ steps.cache_timestamp.outputs.timestamp }}
${{ matrix.os }}-${{ matrix.configuration }}-ccache-
- name: Build and test - name: Build and test
env: env:
...@@ -266,6 +346,23 @@ jobs: ...@@ -266,6 +346,23 @@ jobs:
-DCMAKE_SHARED_LINKER_FLAGS='-fuse-ld=lld' -DCMAKE_SHARED_LINKER_FLAGS='-fuse-ld=lld'
${{ github.workspace }}/cget/bin/ccache -s ${{ github.workspace }}/cget/bin/ccache -s
# GH actions can not update existing cache, as a workaround clear cache and then save it
- name: Clear ccache cache before saving
if: ${{ steps.ccache_restore.outputs.cache-hit }}
shell: bash
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
gh extension install actions/gh-actions-cache --pin v1.0.1
gh actions-cache delete ${{ steps.ccache_restore.outputs.cache-matched-key }} --confirm
- name: Save cache files for ccache
uses: actions/cache/save@v3
if: always()
with:
path: ${{ github.workspace }}/ccache
key: ${{ matrix.os }}-${{ matrix.configuration }}-ccache-${{ github.ref }}
- name: Upload code coverage - name: Upload code coverage
if: "matrix.configuration == 'codecov'" if: "matrix.configuration == 'codecov'"
env: env:
...@@ -303,12 +400,21 @@ jobs: ...@@ -303,12 +400,21 @@ jobs:
steps: steps:
- name: Free space - name: Free space
run: sudo rm -rf /usr/local/android /usr/share/dotnet /usr/local/share/boost /opt/ghc /usr/local/share/chrom* /usr/share/swift /usr/local/julia* /usr/local/lib/android /usr/local/graalvm /usr/local/aws* /usr/local/lib/heroku uses: jlumbroso/free-disk-space@main
with:
tool-cache: true
android: true
dotnet: true
haskell: true
large-packages: true
swap-storage: true
- uses: actions/checkout@v3 - uses: actions/checkout@v3
- name: Set up Python - name: Set up Python
uses: actions/setup-python@v4 uses: actions/setup-python@v4
with: with:
python-version: 3.7 python-version: 3.7
- name: Cache dependencies - name: Cache dependencies
# Ignore the failure of a step and avoid terminating the job. # Ignore the failure of a step and avoid terminating the job.
continue-on-error: true continue-on-error: true
...@@ -317,9 +423,8 @@ jobs: ...@@ -317,9 +423,8 @@ jobs:
# This path is specific to Ubuntu # This path is specific to Ubuntu
path: ${{ github.workspace }}/cget path: ${{ github.workspace }}/cget
# Look to see if there is a cache hit for the corresponding requirements file # Look to see if there is a cache hit for the corresponding requirements file
key: key: ${{ matrix.os }}-cget-4-${{ hashFiles('requirements.txt', 'dev-requirements.txt') }}
${{ matrix.os }}-cget-4-${{ hashFiles('requirements.txt', 'dev-requirements.txt') }} restore-keys: ${{ matrix.os }}-cget-4-
${{ matrix.os }}-cget-4-
- name: Install dependencies - name: Install dependencies
...@@ -328,21 +433,14 @@ jobs: ...@@ -328,21 +433,14 @@ jobs:
pip install https://github.com/RadeonOpenCompute/rbuild/archive/master.tar.gz pip install https://github.com/RadeonOpenCompute/rbuild/archive/master.tar.gz
rbuild prepare -d cget -s gh rbuild prepare -d cget -s gh
sudo apt-get install -y lld sudo apt-get install -y lld
- name: Prepare timestamp
id: cache_timestamp
shell: bash
run: echo timestamp="$(date +'%Y-%m-%dT%H:%M:%S')" >> $GITHUB_OUTPUT
- name: Cache files for ccache - name: Restore cache files for ccache
# Ignore the failure of a step and avoid terminating the job. id: ccache_restore_fpga
continue-on-error: true uses: actions/cache/restore@v3
uses: pat-s/always-upload-cache@v2.1.3
with: with:
path: ccache path: ${{ github.workspace }}/ccache
key: ${{ matrix.os }}-${{ matrix.configuration }}-ccache-${{ steps.cache_timestamp.outputs.timestamp }} key: ${{ matrix.os }}-${{ matrix.configuration }}-ccache-${{ github.ref }}
restore-keys: | restore-keys: ${{ matrix.os }}-${{ matrix.configuration }}-ccache-
${{ matrix.os }}-${{ matrix.configuration }}-ccache-${{ steps.cache_timestamp.outputs.timestamp }}
${{ matrix.os }}-${{ matrix.configuration }}-ccache-
- name: Build and test - name: Build and test
env: env:
...@@ -363,6 +461,24 @@ jobs: ...@@ -363,6 +461,24 @@ jobs:
-DMIGRAPHX_ENABLE_FPGA=On -DMIGRAPHX_ENABLE_FPGA=On
${{ github.workspace }}/cget/bin/ccache -s ${{ github.workspace }}/cget/bin/ccache -s
# this is a workaround, with GH actions can not update existing cache
- name: Clear ccache cache before saving
if: ${{ steps.ccache_restore_fpga.outputs.cache-hit }}
shell: bash
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
gh extension install actions/gh-actions-cache
gh actions-cache delete ${{ steps.ccache_restore_fpga.outputs.cache-matched-key }} --confirm
continue-on-error: true
- name: Save cache files for ccache
uses: actions/cache/save@v3
if: always()
with:
path: ${{ github.workspace }}/ccache
key: ${{ matrix.os }}-${{ matrix.configuration }}-ccache-${{ github.ref }}
#- name: Upload code coverage #- name: Upload code coverage
# if: "matrix.configuration == 'codecov'" # if: "matrix.configuration == 'codecov'"
# env: # env:
...@@ -377,3 +493,4 @@ jobs: ...@@ -377,3 +493,4 @@ jobs:
# chmod +x codecov # chmod +x codecov
# ./codecov -t ${CODECOV_TOKEN} # ./codecov -t ${CODECOV_TOKEN}
# echo "Uploaded" # echo "Uploaded"
.github/workflows/clean-closed-pr-caches.yaml 0 → 100644
View file @ cd4ab535
name: Cleanup caches of closed PR
on:
pull_request:
types:
- closed
jobs:
cleanup:
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v3
- name: Cleanup
run: |
gh extension install actions/gh-actions-cache --pin v1.0.1
REPO=${{ github.repository }}
BRANCH="refs/pull/${{ github.event.pull_request.number }}/merge"
echo "Fetching list of cache key"
cacheKeysForPR=$(gh actions-cache list -R $REPO -B $BRANCH | cut -f 1 | tail -n +3)
## Setting this to not fail the workflow while deleting cache keys.
set +e
echo "Deleting caches..."
for cacheKey in $cacheKeysForPR
do
gh actions-cache delete $cacheKey -R $REPO -B $BRANCH --confirm
done
echo "Done"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
.github/workflows/history.yaml
View file @ cd4ab535
...@@ -20,6 +20,10 @@ on: ...@@ -20,6 +20,10 @@ on:
description: Repository where benchmark utils are stored description: Repository where benchmark utils are stored
required: true required: true
default: "ROCmSoftwarePlatform/migraphx-benchmark-utils" default: "ROCmSoftwarePlatform/migraphx-benchmark-utils"
organization:
description: Organization based on which location of files will be different
required: true
default: "AMD"
jobs: jobs:
release: release:
...@@ -29,6 +33,7 @@ jobs: ...@@ -29,6 +33,7 @@ jobs:
end_date: ${{ github.event.inputs.end_date || 'yyyy-mm-dd' }} end_date: ${{ github.event.inputs.end_date || 'yyyy-mm-dd' }}
history_repo: ${{ github.event.inputs.history_repo || 'ROCmSoftwarePlatform/migraphx-reports' }} history_repo: ${{ github.event.inputs.history_repo || 'ROCmSoftwarePlatform/migraphx-reports' }}
benchmark_utils_repo: ${{ github.event.inputs.benchmark_utils_repo || 'ROCmSoftwarePlatform/migraphx-benchmark-utils' }} benchmark_utils_repo: ${{ github.event.inputs.benchmark_utils_repo || 'ROCmSoftwarePlatform/migraphx-benchmark-utils' }}
organization: ${{ github.event.inputs.organization || 'AMD' }}
secrets: secrets:
gh_token: ${{ secrets.MIGRAPHX_BOT_TOKEN }} gh_token: ${{ secrets.MIGRAPHX_BOT_TOKEN }}
mail_user: ${{ secrets.MAIL_USERNAME }} mail_user: ${{ secrets.MAIL_USERNAME }}
......
.github/workflows/performance.yaml
View file @ cd4ab535
...@@ -12,7 +12,7 @@ on: ...@@ -12,7 +12,7 @@ on:
rocm_release: rocm_release:
description: ROCm Version description: ROCm Version
required: true required: true
default: '5.4.2' default: '5.5'
performance_reports_repo: performance_reports_repo:
description: Repository where performance reports are stored description: Repository where performance reports are stored
required: true required: true
...@@ -29,6 +29,14 @@ on: ...@@ -29,6 +29,14 @@ on:
description: Last N results description: Last N results
required: true required: true
default: '10' default: '10'
model_timeout:
description: If model in performance test script passes this threshold, it will be skipped
required: true
default: '30m'
performance_backup_repo:
description: Repository for backup
required: true
default: migraphx-benchmark/performance-backup
flags: flags:
description: -m for Max value; -s for Std dev; -r for Threshold file description: -m for Max value; -s for Std dev; -r for Threshold file
required: true required: true
...@@ -40,12 +48,14 @@ jobs: ...@@ -40,12 +48,14 @@ jobs:
release: release:
uses: ROCmSoftwarePlatform/migraphx-benchmark/.github/workflows/perf-test.yml@main uses: ROCmSoftwarePlatform/migraphx-benchmark/.github/workflows/perf-test.yml@main
with: with:
rocm_release: ${{ github.event.inputs.rocm_release || '5.4.2' }} rocm_release: ${{ github.event.inputs.rocm_release || '5.5' }}
result_number: ${{ github.event.inputs.result_number || '10' }} result_number: ${{ github.event.inputs.result_number || '10' }}
flags: ${{ github.event.inputs.flags || '-r' }} flags: ${{ github.event.inputs.flags || '-r' }}
performance_reports_repo: ${{ github.event.inputs.performance_reports_repo || 'ROCmSoftwarePlatform/migraphx-reports' }} performance_reports_repo: ${{ github.event.inputs.performance_reports_repo || 'ROCmSoftwarePlatform/migraphx-reports' }}
performance_backup_repo: ${{ github.event.inputs.performance_backup_repo || 'migraphx-benchmark/performance-backup' }}
benchmark_utils_repo: ${{ github.event.inputs.benchmark_utils_repo || 'ROCmSoftwarePlatform/migraphx-benchmark-utils' }} benchmark_utils_repo: ${{ github.event.inputs.benchmark_utils_repo || 'ROCmSoftwarePlatform/migraphx-benchmark-utils' }}
organization: ${{ github.event.inputs.organization || 'AMD' }} organization: ${{ github.event.inputs.organization || 'AMD' }}
model_timeout: ${{ github.event.inputs.model_timeout || '30m' }}
secrets: secrets:
gh_token: ${{ secrets.MIGRAPHX_BOT_TOKEN }} gh_token: ${{ secrets.MIGRAPHX_BOT_TOKEN }}
mail_user: ${{ secrets.MAIL_USERNAME }} mail_user: ${{ secrets.MAIL_USERNAME }}
......
.github/workflows/rocm-image-release.yaml
View file @ cd4ab535
...@@ -10,12 +10,32 @@ on: ...@@ -10,12 +10,32 @@ on:
description: Repository for benchmark utils description: Repository for benchmark utils
required: true required: true
default: 'ROCmSoftwarePlatform/migraphx-benchmark-utils' default: 'ROCmSoftwarePlatform/migraphx-benchmark-utils'
base_image:
description: Base image for rocm Docker build
required: true
default: "rocm/dev-ubuntu-20.04"
docker_image:
description: Docker image name for rocm Docker build
required: true
default: "rocm-migraphx"
build_navi:
description: Build navi number
required: true
default: "0"
overwrite:
type: boolean
description: Overwrite image if it already exists
required: true
jobs: jobs:
release: release:
uses: ROCmSoftwarePlatform/migraphx-benchmark/.github/workflows/rocm-release.yml@main uses: ROCmSoftwarePlatform/migraphx-benchmark/.github/workflows/rocm-release.yml@main
with: with:
rocm_release: ${{ github.event.inputs.rocm_release }} rocm_release: ${{ github.event.inputs.rocm_release || '5.1' }}
benchmark-utils_repo: ${{ github.event.inputs.benchmark-utils_repo || 'ROCmSoftwarePlatform/migraphx-benchmark-utils' }} benchmark-utils_repo: ${{ github.event.inputs.benchmark-utils_repo || 'ROCmSoftwarePlatform/migraphx-benchmark-utils' }}
base_image: ${{ github.event.inputs.base_image || 'rocm/dev-ubuntu-20.04' }}
docker_image: ${{ github.event.inputs.docker_image || 'rocm-migraphx' }}
build_navi: ${{ github.event.inputs.build_navi || '0' }}
overwrite: ${{ github.event.inputs.overwrite == 'true' }}
secrets: secrets:
gh_token: ${{ secrets.MIGRAPHX_BOT_TOKEN }} gh_token: ${{ secrets.MIGRAPHX_BOT_TOKEN }}
.github/workflows/sync-onnxrt-main.yaml
View file @ cd4ab535
name: Onnxruntime main weekly sync name: Onnxruntime main weekly sync
on: on:
schedule: schedule:
- cron: '07 17 * * 5' - cron: '07 17 * * 5'
......
.gitignore
View file @ cd4ab535
...@@ -48,6 +48,8 @@ ...@@ -48,6 +48,8 @@
/CMakeSettings.json /CMakeSettings.json
# documentation artifacts
_toc.yml
#==============================================================================# #==============================================================================#
# Directories to ignore (do not add trailing '/'s, they skip symlinks). # Directories to ignore (do not add trailing '/'s, they skip symlinks).
...@@ -61,3 +63,12 @@ test/onnx/models ...@@ -61,3 +63,12 @@ test/onnx/models
# VS2017 and VSCode config files. # VS2017 and VSCode config files.
.vscode .vscode
.vs .vs
# documentation artifacts
docs/_build
docs/_images
docs/_static
docs/_templates
docs/.doxygen/docBin
docs/_doxygen
/_readthedocs
.readthedocs.yaml 0 → 100644
View file @ cd4ab535
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
version: 2
sphinx:
configuration: docs/conf.py
formats: [htmlzip]
python:
version: "3.8"
install:
- requirements: docs/.sphinx/requirements.txt
CHANGELOG.md
View file @ cd4ab535
# Change Log for MIGraphX # Change Log for MIGraphX
Full documentation for MIGraphX is available at [MIGraphX Github Pages](https://rocmsoftwareplatform.github.io/AMDMIGraphX/doc/html/). Full documentation for MIGraphX is available at [MIGraphX Documentation](https://rocmdocs.amd.com/projects/AMDMIGraphX/en/latest/).
## MIGraphX 2.5 for ROCm 5.5.0 ## MIGraphX 2.5 for ROCm 5.5.0
......
CMakeLists.txt
View file @ cd4ab535
...@@ -69,7 +69,7 @@ include(ROCMSetupVersion) ...@@ -69,7 +69,7 @@ include(ROCMSetupVersion)
option(BUILD_DEV "Build for development purpose only" OFF) option(BUILD_DEV "Build for development purpose only" OFF)
rocm_setup_version(VERSION 2.6.0) rocm_setup_version(VERSION 2.7.0)
set(MIGRAPHX_SO_VERSION ${PROJECT_VERSION_MAJOR}.${PROJECT_VERSION_MINOR}.${PROJECT_VERSION_PATCH}) set(MIGRAPHX_SO_VERSION ${PROJECT_VERSION_MAJOR}.${PROJECT_VERSION_MINOR}.${PROJECT_VERSION_PATCH})
option( BUILD_SHARED_LIBS "Build as a shared library" ON ) option( BUILD_SHARED_LIBS "Build as a shared library" ON )
...@@ -151,6 +151,8 @@ rocm_enable_clang_tidy( ...@@ -151,6 +151,8 @@ rocm_enable_clang_tidy(
-clang-diagnostic-extern-c-compat -clang-diagnostic-extern-c-compat
-clang-diagnostic-disabled-macro-expansion -clang-diagnostic-disabled-macro-expansion
-clang-diagnostic-unused-command-line-argument -clang-diagnostic-unused-command-line-argument
-cppcoreguidelines-avoid-do-while
-cppcoreguidelines-avoid-const-or-ref-data-members
-cppcoreguidelines-explicit-virtual-functions -cppcoreguidelines-explicit-virtual-functions
-cppcoreguidelines-init-variables -cppcoreguidelines-init-variables
-cppcoreguidelines-pro-bounds-array-to-pointer-decay -cppcoreguidelines-pro-bounds-array-to-pointer-decay
...@@ -198,7 +200,6 @@ rocm_enable_clang_tidy( ...@@ -198,7 +200,6 @@ rocm_enable_clang_tidy(
EXTRA_ARGS EXTRA_ARGS
-UNDEBUG -UNDEBUG
-DMIGRAPHX_USE_CLANG_TIDY -DMIGRAPHX_USE_CLANG_TIDY
"-Dmain\\\\(...\\\\)=main\\\\(__VA_ARGS__\\\\) // NOLINT"
CLANG_ARGS CLANG_ARGS
-analyzer-max-loop 10 -analyzer-max-loop 10
-analyzer-inline-max-stack-depth 10 -analyzer-inline-max-stack-depth 10
...@@ -267,7 +268,7 @@ set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/lib) ...@@ -267,7 +268,7 @@ set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/lib)
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/lib) set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/lib)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/bin) set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/bin)
add_subdirectory(src) add_subdirectory(src)
add_subdirectory(doc) add_subdirectory(docs)
add_subdirectory(test) add_subdirectory(test)
add_subdirectory(tools) add_subdirectory(tools)
......
Dockerfile
View file @ cd4ab535
...@@ -10,7 +10,7 @@ RUN apt-get update && apt-get install -y gnupg2 --no-install-recommends curl && ...@@ -10,7 +10,7 @@ RUN apt-get update && apt-get install -y gnupg2 --no-install-recommends curl &&
curl -sL http://repo.radeon.com/rocm/rocm.gpg.key | apt-key add - curl -sL http://repo.radeon.com/rocm/rocm.gpg.key | apt-key add -
# Add rocm repository # Add rocm repository
RUN sh -c 'echo deb [arch=amd64 trusted=yes] http://repo.radeon.com/rocm/apt/5.4.2/ ubuntu main > /etc/apt/sources.list.d/rocm.list' RUN sh -c 'echo deb [arch=amd64 trusted=yes] http://repo.radeon.com/rocm/apt/5.5/ focal main > /etc/apt/sources.list.d/rocm.list'
# Install dependencies # Install dependencies
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated \ RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-unauthenticated \
...@@ -83,7 +83,7 @@ RUN test -f /usr/local/hash || exit 1 ...@@ -83,7 +83,7 @@ RUN test -f /usr/local/hash || exit 1
RUN pip3 install yapf==0.28.0 RUN pip3 install yapf==0.28.0
# Install doc requirements # Install doc requirements
ADD doc/requirements.txt /doc-requirements.txt ADD docs/.sphinx/requirements.txt /doc-requirements.txt
RUN pip3 install -r /doc-requirements.txt RUN pip3 install -r /doc-requirements.txt
# Download real models to run onnx unit tests # Download real models to run onnx unit tests
...@@ -110,7 +110,7 @@ RUN git clone --single-branch --branch ${ONNXRUNTIME_BRANCH} --recursive ${ONNXR ...@@ -110,7 +110,7 @@ RUN git clone --single-branch --branch ${ONNXRUNTIME_BRANCH} --recursive ${ONNXR
ADD tools/build_and_test_onnxrt.sh /onnxruntime/build_and_test_onnxrt.sh ADD tools/build_and_test_onnxrt.sh /onnxruntime/build_and_test_onnxrt.sh
RUN cget -p /usr/local install ROCmSoftwarePlatform/rocMLIR@78b706fe9879587ab98b6614ae539265374a3fae -DBUILD_MIXR_TARGET=On -DLLVM_ENABLE_ZSTD=Off -DLLVM_ENABLE_THREADS=Off RUN cget -p /usr/local install ROCmSoftwarePlatform/rocMLIR@a997d5f51314b45d7a4c04f1599966dcf53f9b4d -DBUILD_MIXR_TARGET=On -DLLVM_ENABLE_ZSTD=Off -DLLVM_ENABLE_THREADS=Off
ENV MIOPEN_FIND_DB_PATH=/tmp/miopen/find-db ENV MIOPEN_FIND_DB_PATH=/tmp/miopen/find-db
ENV MIOPEN_USER_DB_PATH=/tmp/miopen/user-db ENV MIOPEN_USER_DB_PATH=/tmp/miopen/user-db
...@@ -120,4 +120,3 @@ ENV LD_LIBRARY_PATH=$PREFIX/lib ...@@ -120,4 +120,3 @@ ENV LD_LIBRARY_PATH=$PREFIX/lib
ENV UBSAN_OPTIONS=print_stacktrace=1 ENV UBSAN_OPTIONS=print_stacktrace=1
ENV ASAN_OPTIONS=detect_stack_use_after_return=1:check_initialization_order=1:strict_init_order=1 ENV ASAN_OPTIONS=detect_stack_use_after_return=1:check_initialization_order=1:strict_init_order=1
RUN ln -s /opt/rocm/llvm/bin/llvm-symbolizer /usr/bin/llvm-symbolizer RUN ln -s /opt/rocm/llvm/bin/llvm-symbolizer /usr/bin/llvm-symbolizer
Jenkinsfile
View file @ cd4ab535
...@@ -29,7 +29,13 @@ def rocmtestnode(Map conf) { ...@@ -29,7 +29,13 @@ def rocmtestnode(Map conf) {
mkdir build mkdir build
cd build cd build
cmake -DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER_LAUNCHER=ccache -DBUILD_DEV=On ${flags} .. cmake -DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER_LAUNCHER=ccache -DBUILD_DEV=On ${flags} ..
make -j\$(nproc) generate all doc package check VERBOSE=1 git diff
git diff-index --quiet HEAD || (echo "Git repo is not clean after running cmake." && exit 1)
make -j\$(nproc) generate VERBOSE=1
git diff
git diff-index --quiet HEAD || (echo "Generated files are different. Please run make generate and commit the changes." && exit 1)
make -j\$(nproc) all doc package check VERBOSE=1
md5sum ./*.deb
""" """
echo cmd echo cmd
sh cmd sh cmd
...@@ -83,8 +89,10 @@ def rocmnodename(name) { ...@@ -83,8 +89,10 @@ def rocmnodename(name) {
node_name = "${rocmtest_name} && vega"; node_name = "${rocmtest_name} && vega";
} else if(name == "navi21") { } else if(name == "navi21") {
node_name = "${rocmtest_name} && navi21"; node_name = "${rocmtest_name} && navi21";
} else if(name == "anygpu") {
node_name = "${rocmtest_name} && (gfx908 || gfx90a || vega)";
} else if(name == "nogpu") { } else if(name == "nogpu") {
return rocmtest_name; node_name = "${rocmtest_name} && nogpu";
} }
return node_name return node_name
} }
...@@ -114,6 +122,10 @@ rocmtest clang_debug: rocmnode('vega') { cmake_build -> ...@@ -114,6 +122,10 @@ rocmtest clang_debug: rocmnode('vega') { cmake_build ->
stage('HipRTC GPU Debug') { stage('HipRTC GPU Debug') {
cmake_build(flags: "-DCMAKE_BUILD_TYPE=release -DMIGRAPHX_USE_HIPRTC=On", gpu_debug: true, hiprtc_workarounds: true) cmake_build(flags: "-DCMAKE_BUILD_TYPE=release -DMIGRAPHX_USE_HIPRTC=On", gpu_debug: true, hiprtc_workarounds: true)
} }
}, all_targets_debug : rocmnode('vega') { cmake_build ->
stage('All targets Release') {
cmake_build(flags: "-DCMAKE_BUILD_TYPE=release -DMIGRAPHX_ENABLE_GPU=On -DMIGRAPHX_ENABLE_CPU=On -DMIGRAPHX_ENABLE_FPGA=On")
}
}, mlir_debug: rocmnode('vega') { cmake_build -> }, mlir_debug: rocmnode('vega') { cmake_build ->
stage('MLIR Debug') { stage('MLIR Debug') {
withEnv(['MIGRAPHX_ENABLE_MLIR=1']) { withEnv(['MIGRAPHX_ENABLE_MLIR=1']) {
...@@ -143,11 +155,12 @@ def onnxnode(name, body) { ...@@ -143,11 +155,12 @@ def onnxnode(name, body) {
} }
} }
rocmtest onnx: onnxnode('rocmtest') { cmake_build -> rocmtest onnx: onnxnode('anygpu') { cmake_build ->
stage("Onnx runtime") { stage("Onnx runtime") {
sh ''' sh '''
apt install half apt install half
ls -lR #ls -lR
md5sum ./build/*.deb
dpkg -i ./build/*.deb dpkg -i ./build/*.deb
cd /onnxruntime && ./build_and_test_onnxrt.sh cd /onnxruntime && ./build_and_test_onnxrt.sh
''' '''
......
README.md
View file @ cd4ab535
...@@ -56,7 +56,7 @@ build MIGraphX. The specific steps are as follows: ...@@ -56,7 +56,7 @@ build MIGraphX. The specific steps are as follows:
1) Install rocm-cmake, pip3, rocblas, and miopen-hip with the command 1) Install rocm-cmake, pip3, rocblas, and miopen-hip with the command
``` ```
sudo apt update && sudo apt install -y rocm-cmake python3-pip rocblas miopen-hip sudo apt install -y rocm-cmake python3-pip rocblas miopen-hip
``` ```
2) Install [rbuild](https://github.com/RadeonOpenCompute/rbuild) (sudo may be required here.) 2) Install [rbuild](https://github.com/RadeonOpenCompute/rbuild) (sudo may be required here.)
...@@ -68,14 +68,11 @@ pip3 install https://github.com/RadeonOpenCompute/rbuild/archive/master.tar.gz ...@@ -68,14 +68,11 @@ pip3 install https://github.com/RadeonOpenCompute/rbuild/archive/master.tar.gz
3) Build MIGraphX source code 3) Build MIGraphX source code
``` ```
rbuild build -d depend -B build --cxx=/opt/rocm/llvm/bin/clang++ rbuild build -d depend -B build
``` ```
then all the prerequisites are in the folder `depend`, and MIGraphX is built in the `build` directory. then all the prerequisites are in the folder `depend`, and MIGraphX is built in the `build` directory.
Note that for ROCm3.7 and later releases, Ubuntu 18.04 or later releases are needed.
Upgrade to Ubuntu 18.04 is available at [Upgrade Ubuntu to 18.04](https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/wiki/Upgrade-to-Ubuntu-18.04-for-ROCM3.7-or-later-releases)
Also note that you may meet the error of `rbuild: command not found`. It is because rbuild is installed Also note that you may meet the error of `rbuild: command not found`. It is because rbuild is installed
at `$HOME/.local/bin`, which is not in `PATH`. You can either export PATH as `export PATH=$HOME/.local/bin:$PATH` at `$HOME/.local/bin`, which is not in `PATH`. You can either export PATH as `export PATH=$HOME/.local/bin:$PATH`
to add the folder to `PATH` or add the option `--prefix /usr/local` in the pip3 command when installing rbuild. to add the folder to `PATH` or add the option `--prefix /usr/local` in the pip3 command when installing rbuild.
...@@ -89,7 +86,7 @@ If using this approach, we need to install the prerequisites, configure the cmak ...@@ -89,7 +86,7 @@ If using this approach, we need to install the prerequisites, configure the cmak
For convenience, the prerequisites can be built automatically with rbuild as: For convenience, the prerequisites can be built automatically with rbuild as:
``` ```
rbuild build -d depend --cxx=/opt/rocm/llvm/bin/clang++ rbuild prepare -d depend
``` ```
then all the prerequisites are in the folder `depend`, and they can be used in the `cmake` configuration then all the prerequisites are in the folder `depend`, and they can be used in the `cmake` configuration
...@@ -174,7 +171,6 @@ To install: ...@@ -174,7 +171,6 @@ To install:
dpkg -i <path_to_deb_file> dpkg -i <path_to_deb_file>
``` ```
### Calling MIGraphX APIs ### Calling MIGraphX APIs
To use MIGraphX's C/C++ API in your cmake project, we need to set `CMAKE_PREFIX_PATH` to the MIGraphX To use MIGraphX's C/C++ API in your cmake project, we need to set `CMAKE_PREFIX_PATH` to the MIGraphX
installation location and then do installation location and then do
...@@ -184,20 +180,42 @@ target_link_libraries(myApp migraphx::c) ...@@ -184,20 +180,42 @@ target_link_libraries(myApp migraphx::c)
``` ```
Where `myApp` is the cmake target in your project. Where `myApp` is the cmake target in your project.
## Building for development
Using rbuild, the dependencies for development can be installed with:
```
rbuild develop
```
This will install the dependencies for development into the `deps` directory and
configure `cmake` to use those dependencies in the `build` directory. These
directories can be changed by passing the `--deps-dir` and `--build-dir` flags
to `rbuild` command:
### Building the documentation ```
rbuild develop --build-dir build_rocm_55 --deps-dir /home/user/deps_dir
```
## Building the documentation
HTML and PDF documentation can be built using: HTML and PDF documentation can be built using:
`cmake --build . --config Release --target doc` **OR** `make doc` `cmake --build . --config Release --target doc` **OR** `make doc`
This will build a local searchable web site inside the doc/html folder. This will build a local searchable web site inside the docs/html folder.
Documentation is built using [Doxygen](http://www.stack.nl/~dimitri/doxygen/download.html), [Sphinx](http://www.sphinx-doc.org/en/stable/index.html), and [Breathe](https://breathe.readthedocs.io/en/latest/) Documentation is built using [Doxygen](http://www.stack.nl/~dimitri/doxygen/download.html) and [rocm-docs-core](https://github.com/RadeonOpenCompute/rocm-docs-core)
Requirements for both Sphinx and Breathe can be installed with: Run the steps below to build documentation locally.
`pip install -r doc/requirements.txt` ```
cd docs
pip3 install -r .sphinx/requirements.txt
python3 -m sphinx -T -E -b html -d _build/doctrees -D language=en . _build/html
```
Depending on your setup `sudo` may be required for the pip install. Depending on your setup `sudo` may be required for the pip install.
......
cmake/Embed.cmake
View file @ cd4ab535
...@@ -24,6 +24,40 @@ ...@@ -24,6 +24,40 @@
find_program(EMBED_LD ld) find_program(EMBED_LD ld)
find_program(EMBED_OBJCOPY objcopy) find_program(EMBED_OBJCOPY objcopy)
if(LINUX)
option(EMBED_USE_LD "Use ld to embed data files" ON)
else()
option(EMBED_USE_LD "Use ld to embed data files" OFF)
endif()
function(wrap_string)
set(options)
set(oneValueArgs VARIABLE AT_COLUMN)
set(multiValueArgs)
cmake_parse_arguments(PARSE "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})
cmake_parse_arguments(WRAP_STRING "${options}" "${oneValueArgs}" "" ${ARGN})
string(LENGTH ${${PARSE_VARIABLE}} string_length)
math(EXPR offset "0")
while(string_length GREATER 0)
if(string_length GREATER ${PARSE_AT_COLUMN})
math(EXPR length "${PARSE_AT_COLUMN}")
else()
math(EXPR length "${string_length}")
endif()
string(SUBSTRING ${${PARSE_VARIABLE}} ${offset} ${length} line)
set(lines "${lines}\n${line}")
math(EXPR string_length "${string_length} - ${length}")
math(EXPR offset "${offset} + ${length}")
endwhile()
set(${PARSE_VARIABLE} "${lines}" PARENT_SCOPE)
endfunction()
function(generate_embed_source EMBED_NAME) function(generate_embed_source EMBED_NAME)
set(options) set(options)
set(oneValueArgs SRC HEADER) set(oneValueArgs SRC HEADER)
...@@ -46,14 +80,21 @@ function(generate_embed_source EMBED_NAME) ...@@ -46,14 +80,21 @@ function(generate_embed_source EMBED_NAME)
list(GET PARSE_OBJECTS ${idx} OBJECT) list(GET PARSE_OBJECTS ${idx} OBJECT)
set(START_SYMBOL "_binary_${SYMBOL}_start") set(START_SYMBOL "_binary_${SYMBOL}_start")
set(END_SYMBOL "_binary_${SYMBOL}_end") set(END_SYMBOL "_binary_${SYMBOL}_end")
if(EMBED_USE_LD)
string(APPEND EXTERNS " string(APPEND EXTERNS "
extern const char ${START_SYMBOL}[]; extern const char ${START_SYMBOL}[];
extern const char ${END_SYMBOL}[]; extern const char ${END_SYMBOL}[];
") ")
else()
string(APPEND EXTERNS "
extern const char ${START_SYMBOL}[];
extern const char* ${END_SYMBOL};
")
endif()
# TODO: Should use NAME_WLE # TODO: Should use NAME_WLE
get_filename_component(BASE_NAME "${OBJECT}" NAME) get_filename_component(BASE_NAME "${OBJECT}" NAME)
string(REGEX REPLACE ".[A-Za-z0-9_]$" "" BASE_NAME ${BASE_NAME}) string(REGEX REPLACE ".[A-Za-z0-9_]+$" "" BASE_NAME ${BASE_NAME})
string(APPEND INIT_KERNELS " string(APPEND INIT_KERNELS "
{ \"${BASE_NAME}\", { ${START_SYMBOL}, ${END_SYMBOL}} }, { \"${BASE_NAME}\", { ${START_SYMBOL}, ${END_SYMBOL}} },
...@@ -86,9 +127,14 @@ function(embed_file OUTPUT_FILE OUTPUT_SYMBOL FILE) ...@@ -86,9 +127,14 @@ function(embed_file OUTPUT_FILE OUTPUT_SYMBOL FILE)
string(MAKE_C_IDENTIFIER "${REL_FILE}" SYMBOL) string(MAKE_C_IDENTIFIER "${REL_FILE}" SYMBOL)
get_filename_component(OUTPUT_FILE_DIR "${REL_FILE}" DIRECTORY) get_filename_component(OUTPUT_FILE_DIR "${REL_FILE}" DIRECTORY)
file(MAKE_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/${OUTPUT_FILE_DIR}") file(MAKE_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/${OUTPUT_FILE_DIR}")
if(EMBED_USE_LD)
set(OUT_FILE "${CMAKE_CURRENT_BINARY_DIR}/${REL_FILE}.o") set(OUT_FILE "${CMAKE_CURRENT_BINARY_DIR}/${REL_FILE}.o")
else()
set(OUT_FILE "${CMAKE_CURRENT_BINARY_DIR}/${REL_FILE}.cpp")
endif()
set(${OUTPUT_SYMBOL} ${SYMBOL} PARENT_SCOPE) set(${OUTPUT_SYMBOL} ${SYMBOL} PARENT_SCOPE)
set(${OUTPUT_FILE} "${OUT_FILE}" PARENT_SCOPE) set(${OUTPUT_FILE} "${OUT_FILE}" PARENT_SCOPE)
if(EMBED_USE_LD)
add_custom_command( add_custom_command(
OUTPUT "${OUT_FILE}" OUTPUT "${OUT_FILE}"
COMMAND ${EMBED_LD} -r -o "${OUT_FILE}" -z noexecstack --format=binary "${REL_FILE}" COMMAND ${EMBED_LD} -r -o "${OUT_FILE}" -z noexecstack --format=binary "${REL_FILE}"
...@@ -97,6 +143,21 @@ function(embed_file OUTPUT_FILE OUTPUT_SYMBOL FILE) ...@@ -97,6 +143,21 @@ function(embed_file OUTPUT_FILE OUTPUT_SYMBOL FILE)
DEPENDS ${FILE} DEPENDS ${FILE}
VERBATIM VERBATIM
) )
else()
set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS ${FILE})
# reads source file contents as hex string
file(READ ${FILE} HEX_STRING HEX)
# wraps the hex string into multiple lines
wrap_string(VARIABLE HEX_STRING AT_COLUMN 80)
# adds '0x' prefix and comma suffix before and after every byte respectively
string(REGEX REPLACE "([0-9a-f][0-9a-f])" "0x\\1, " ARRAY_VALUES ${HEX_STRING})
# removes trailing comma
string(REGEX REPLACE ", $" "" ARRAY_VALUES ${ARRAY_VALUES})
file(WRITE "${OUT_FILE}" "
extern const char _binary_${SYMBOL}_start[] = { ${ARRAY_VALUES} };
extern const char* _binary_${SYMBOL}_end = _binary_${SYMBOL}_start + sizeof(_binary_${SYMBOL}_start);
\n")
endif()
endforeach() endforeach()
endfunction() endfunction()
...@@ -119,6 +180,6 @@ function(add_embed_library EMBED_NAME) ...@@ -119,6 +180,6 @@ function(add_embed_library EMBED_NAME)
generate_embed_source(${EMBED_NAME} SRC ${SRC_FILE} HEADER ${HEADER_FILE} OBJECTS ${OUTPUT_FILES} SYMBOLS ${SYMBOLS}) generate_embed_source(${EMBED_NAME} SRC ${SRC_FILE} HEADER ${HEADER_FILE} OBJECTS ${OUTPUT_FILES} SYMBOLS ${SYMBOLS})
add_library(${EMBED_NAME} STATIC ${OUTPUT_FILES} "${SRC_FILE}") add_library(${EMBED_NAME} STATIC ${OUTPUT_FILES} "${SRC_FILE}")
target_include_directories(${EMBED_NAME} PUBLIC "${EMBED_DIR}/include") target_include_directories(${EMBED_NAME} PUBLIC "${EMBED_DIR}/include")
target_compile_options(${EMBED_NAME} PRIVATE -Wno-reserved-identifier) target_compile_options(${EMBED_NAME} PRIVATE -Wno-reserved-identifier -Wno-extern-initializer -Wno-missing-variable-declarations)
set_target_properties(${EMBED_NAME} PROPERTIES POSITION_INDEPENDENT_CODE On) set_target_properties(${EMBED_NAME} PROPERTIES POSITION_INDEPENDENT_CODE On)
endfunction() endfunction()
doc/src/conf.py deleted 100755 → 0
View file @ 3891ee58
#####################################################################################
# The MIT License (MIT)
#
# Copyright (c) 2015-2022 Advanced Micro Devices, Inc. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
#####################################################################################
# -*- coding: utf-8 -*-
#
# MIGraphX documentation build configuration file, created by
# sphinx-quickstart on Thu Jul 19 11:38:13 2018.
#
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
from datetime import date
import re
# sys.path.insert(0, os.path.abspath('.'))
# -- General configuration ------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#
# needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'breathe', 'sphinx.ext.mathjax', 'sphinx.ext.viewcode', 'sphinx_rtd_theme',
'sphinx.ext.autosectionlabel'
]
autosectionlabel_prefix_document = True
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
#
# source_suffix = ['.rst', '.md']
source_suffix = '.rst'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'MIGraphX'
copyright = u'2018-{}, AMD'.format(date.today().year)
author = u'AMD'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
with open('../../CMakeLists.txt') as file:
version = next((re.findall('[0-9.]+', line)[0]
for line in file.readlines()
if 'rocm_setup_version' in line))
# The full version, including alpha/beta/rc tags.
release = version
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This patterns also effect to html_static_path and html_extra_path
exclude_patterns = []
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
highlight_language = 'cpp'
# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = False
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = 'sphinx_rtd_theme'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#
# html_theme_options = {}
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# -- Options for HTMLHelp output ------------------------------------------
# Output file base name for HTML help builder.
htmlhelp_basename = 'MIGraphXdoc'
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#
# 'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#
# 'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#
# 'preamble': '',
# Latex figure (float) alignment
#
# 'figure_align': 'htbp',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, 'MIGraphX.tex', u'MIGraphX Documentation', u'AMD', 'manual'),
]
# -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [(master_doc, 'migraphx', u'MIGraphX Documentation', [author], 1)]
# -- Options for Texinfo output -------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'MIGraphX', u'MIGraphX Documentation', author, 'MIGraphX',
'One line description of project.', 'Miscellaneous'),
]
breathe_default_members = ('members', 'undoc-members')
cpp_index_common_prefix = ['migraphx::']
default_role = 'any'
primary_domain = 'cpp'
docs/.doxygen/Doxyfile 0 → 100644
View file @ cd4ab535
# Auto-generated doxygen configuration file
BUILTIN_STL_SUPPORT = YES
CALLER_GRAPH = YES
CALL_GRAPH = YES
ENUM_VALUES_PER_LINE = 1
EXCLUDE_PATTERNS = ../../src/targets/gpu/kernels ../../src/targets/gpu/device
EXTRACT_ALL = YES
FULL_PATH_NAMES = YES
GENERATE_LATEX = NO
GENERATE_TREEVIEW = YES
GENERATE_XML = YES
INCLUDE_PATH = ../../src/include ../../src/targets/cpu/include ../../src/targets/gpu/include
INPUT = ../../src
MACRO_EXPANSION = YES
OUTPUT_DIRECTORY = docBin
PREDEFINED = DOXYGEN
PROJECT_NAME = MIGraphX
RECURSIVE = YES
REFERENCED_BY_RELATION = YES
REFERENCES_LINK_SOURCE = YES
REFERENCES_RELATION = YES
SEARCH_INCLUDES = YES
SORT_MEMBERS_CTORS_1ST = YES
SOURCE_BROWSER = YES
STRIP_FROM_INC_PATH = ../../src/include ../../src/targets/cpu/include ../../src/targets/gpu/include
STRIP_FROM_PATH =
USE_PDFLATEX = NO
WARN_LOGFILE = DoxygenWarningLog.txt
DOT_PATH =
HAVE_DOT = NO
docs/.sphinx/_toc.yml.in 0 → 100644
View file @ cd4ab535
# Anywhere {branch} is used, the branch name will be substituted.
# These comments will also be removed.
root: index
subtrees:
- numbered: False
entries:
- file: py_user_guide
- file: cpp_user_guide
- file: driver
- file: contributor_guide
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment