Unverified Commit 165d1a17 authored by Shucai Xiao's avatar Shucai Xiao Committed by GitHub
Browse files

Refine readme (#749)



* update README.md file

* update README.md file

* update README.md file

* update README.md file

* update README.md file

* update dependencies

* update dependencies

* update readme file

* format issue

* edit

* edit text

* update script to install all dependencies

* remove unnecessary libs

* refine readme file

* refine readme file

* refine readme file

* refine readme file

* refine README

* refine readme.md file

* fix a error in the README.md file

* change to use Dockerfile to call install_prereqs.sh

* refine readme file

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* minor changes

* fixed a bug

* refine readme file

* minor change

* minor changes

* minor changes

* minor changes

* some additional refinement

* fix review comments

* some refinement

* fix review comments

* refine readme file

* fix minor issue

* change the miopen versioin back to 2.4.0

* fix an error in find_package

* remove an unnecessary change

* some additional refinement of the readme file

* fix readme file

* fix a review comments

* added one more note for rbuild

* add more informat for rbuild not found

* add a period

* Update README.md
Co-authored-by: default avatarkahmed10 <15948690+kahmed10@users.noreply.github.com>

* Update README.md
Co-authored-by: default avatarkahmed10 <15948690+kahmed10@users.noreply.github.com>

* Update README.md
Co-authored-by: default avatarkahmed10 <15948690+kahmed10@users.noreply.github.com>

* Update README.md
Co-authored-by: default avatarkahmed10 <15948690+kahmed10@users.noreply.github.com>

* Update README.md
Co-authored-by: default avatarkahmed10 <15948690+kahmed10@users.noreply.github.com>

* Update README.md
Co-authored-by: default avatarkahmed10 <15948690+kahmed10@users.noreply.github.com>

* Update README.md
Co-authored-by: default avatarkahmed10 <15948690+kahmed10@users.noreply.github.com>

* Update README.md
Co-authored-by: default avatarkahmed10 <15948690+kahmed10@users.noreply.github.com>

* refine a link in the readme file

* some additional refinement

* minor changes

* minor changes

* minor changes
Co-authored-by: default avatarkahmed10 <15948690+kahmed10@users.noreply.github.com>
Co-authored-by: default avatarmvermeulen <5479696+mvermeulen@users.noreply.github.com>
parent 867539b7
......@@ -65,21 +65,13 @@ RUN pip3 install onnx==1.7.0 numpy==1.18.5 typing==3.7.4 pytest==6.0.1
# Download real models to run onnx unit tests
ENV ONNX_HOME=$HOME
COPY ./tools/download_models.sh /
RUN chmod +x /download_models.sh && /download_models.sh && rm /download_models.sh
RUN /download_models.sh && rm /download_models.sh
# Install dependencies
ADD dev-requirements.txt /dev-requirements.txt
ADD requirements.txt /requirements.txt
# Manually ignore rocm dependencies
RUN cget -p $PREFIX ignore \
RadeonOpenCompute/clang-ocl \
ROCm-Developer-Tools/HIP \
ROCmSoftwarePlatform/MIOpen \
ROCmSoftwarePlatform/MIOpenGEMM \
ROCmSoftwarePlatform/rocBLAS
RUN cget -p $PREFIX init --cxx /opt/rocm/llvm/bin/clang++
RUN cget -p $PREFIX install -f dev-requirements.txt
RUN cget -p $PREFIX install oneapi-src/oneDNN@v1.7
COPY ./tools/install_prereqs.sh /
RUN /install_prereqs.sh /usr/local / && rm /install_prereqs.sh
# Install latest ccache version
RUN cget -p $PREFIX install facebook/zstd@v1.4.5 -X subdir -DCMAKE_DIR=build/cmake
......@@ -100,9 +92,9 @@ ADD tools/build_and_test_onnxrt.sh /onnxruntime/build_and_test_onnxrt.sh
ENV MIOPEN_FIND_DB_PATH=/tmp/miopen/find-db
ENV MIOPEN_USER_DB_PATH=/tmp/miopen/user-db
ENV LD_LIBRARY_PATH=$PREFIX/lib
# Setup ubsan environment to printstacktrace
ENV UBSAN_OPTIONS=print_stacktrace=1
ENV ASAN_OPTIONS=detect_stack_use_after_return=1:check_initialization_order=1:strict_init_order=1
# AMD MIGraphX
AMD's graph optimization engine.
AMD MIGraphX is AMD's graph inference engine that accelerates machine learning model inference. AMD MIGraphX can be used by
installing binaries directly or building from source code.
In the following, instructions of how to build and install MIGraphX are described with Ubuntu as the OS
(Instructions of installation on other Linux OSes will come later). Note that all the following instructions assume
ROCm has been installed successfully. ROCm installation instructions are explained in the [ROCm installation
guide](https://rocmdocs.amd.com/en/latest/Installation_Guide/Installation-Guide.html).
## Installing from binaries
With ROCm installed correctly, MIGraphX binaries can be installed on Ubuntu with the following command:
```
sudo apt update && sudo apt install -y migraphx
```
then the header files and libs are installed under `/opt/rocm-<version>`, where `<version>` is the ROCm version.
## Building from source
There are three ways to build the MIGraphX sources.
* [Use the ROCm build tool](#use-the-rocm-build-tool-rbuild)
This approach uses [rbuild](https://github.com/RadeonOpenCompute/rbuild) to install the prerequisites and
build the libs with just one command.
* [Use cmake](#use-cmake-to-build-migraphx)
This approach uses a script to install the prerequisites, then use cmake to build the source.
* [Use docker](#use-docker)
This approach builds a docker image with all prerequisites installed, then build the MIGraphX sources inside a docker container.
In the following, we will first list the prerequisites required to build MIGraphX source code, then describe
each of the three approaches.
### List of prerequisites
The following is a list of prerequisites required to build MIGraphX source.
## Prerequisites
* [ROCm cmake modules](https://github.com/RadeonOpenCompute/rocm-cmake) **required**
* [MIOpen](https://github.com/ROCmSoftwarePlatform/MIOpen) for running on the GPU
* [rocBLAS](https://github.com/ROCmSoftwarePlatform/rocBLAS) for running on the GPU
* [HIP](https://github.com/ROCm-Developer-Tools/HIP) for running on the GPU
* [Protobuf](https://github.com/google/protobuf) for reading [onnx](https://github.com/onnx/onnx) files
* [Half](http://half.sourceforge.net/) - IEEE 754-based half-precision floating point library
* [pybind11](https://pybind11.readthedocs.io/en/stable/) - for python bindings
* [JSON](https://github.com/nlohmann/json) - for model serialization to json string format
* [MessagePack](https://msgpack.org/index.html) - for model serialization to binary format
## Installing the dependencies
#### Use the ROCm build tool [rbuild](https://github.com/RadeonOpenCompute/rbuild).
Dependencies can be installed using the ROCm build tool [rbuild](https://github.com/RadeonOpenCompute/rbuild).
In this approach, we use the [rbuild](https://github.com/RadeonOpenCompute/rbuild) build tool to
build MIGraphX. The specific steps are as follows:
1) Install rocm-cmake, pip3, rocblas, and miopen-hip with the command
To install rbuild:
```
pip install https://github.com/RadeonOpenCompute/rbuild/archive/master.tar.gz
sudo apt update && sudo apt install -y rocm-cmake python3-pip rocblas miopen-hip
```
To build dependencies along with MIGraphX
2) Install [rbuild](https://github.com/RadeonOpenCompute/rbuild) (sudo may be required here.)
```
rbuild build -d depend --cxx=/opt/rocm/llvm/bin/clang++
pip3 install https://github.com/RadeonOpenCompute/rbuild/archive/master.tar.gz
```
This builds dependencies in the subdirectory named depend and then builds MIGraphX using these dependencies.
## Building MIGraphX from source
3) Build MIGraphX source code
## Configuring with cmake
```
rbuild build -d depend -B build --cxx=/opt/rocm/llvm/bin/clang++
```
First create a build directory:
then all the prerequisites are in the folder `depend`, and MIGraphX is built in the `build` directory.
Note that for ROCm3.7 and later releases, Ubuntu 18.04 or later releases are needed.
Upgrade to Ubuntu 18.04 is available at [Upgrade Ubuntu to 18.04](https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/wiki/Upgrade-to-Ubuntu-18.04-for-ROCM3.7-or-later-releases)
```
mkdir build;
cd build;
```
Also note that you may meet the error of `rbuild: command not found`. It is because rbuild is installed
at `$HOME/.local/bin`, which is not in `PATH`. You can either export PATH as `export PATH=$HOME/.local/bin:$PATH`
to add the folder to `PATH` or add the option `--prefix /usr/local` in the pip3 command when installing rbuild.
#### Use cmake to build MIGraphX
If using this approach, we need to install the prerequisites, configure the cmake, and then build the source.
Next configure cmake. The hcc compiler is required to build the MIOpen backend:
##### Installing the prerequisites
For convenience, the prerequisites can be built automatically with rbuild as:
```
CXX=/opt/rocm/llvm/bin/clang++ cmake ..
rbuild build -d depend --cxx=/opt/rocm/llvm/bin/clang++
```
If the dependencies from `install_deps.cmake` was installed to another directory, the `CMAKE_PREFIX_PATH` needs to be set to what `--prefix` was set to from `install_deps.cmake`:
then all the prerequisites are in the folder `depend`, and they can be used in the `cmake` configuration
as `-DCMAKE_PREFIX_PATH=depend`.
If you have sudo access, as an alternative to the rbuild command, you can install the prerequisites just
like in the dockerfile by calling `./tools/install_prereqs.sh`.
(Note that this script is for Ubuntu. By default, all prerequisites are installed at the default location `/usr/local`
and are accessible by all users. For the default location, `sudo` is required to run the script.
You can also specify a location at which the prerequisites are installed with `./tools/install_prereqs.sh $your_loc`.)
##### Building MIGraphX source and install libs
With the above prerequisites installed, we can build source as:
1) Go to the project folder and create a `build` directory:
```
CXX=/opt/rocm/llvm/bin/clang++ cmake -DCMAKE_PREFIX_PATH=/some/dir ..
mkdir build
cd build
```
2) Configure the cmake. If the prerequisites are installed at the default location `/usr/local`, the command is:
#### Changing the cmake configuration
```
CXX=/opt/rocm/llvm/bin/clang++ cmake ..
```
Otherwise, you need to set `-DCMAKE_PREFIX_PATH=$your_loc` to configure the cmake.
The configuration can be changed after running cmake by using `ccmake`:
3) Build MIGraphX source code
` ccmake .. ` **OR** `cmake-gui`: ` cmake-gui ..`
```
make -j$(nproc)
```
## Building the library
Correctness can be verified as:
The library can be built, from the `build` directory using the 'Release' configuration:
```
make -j$(nproc) check
```
` cmake --build . --config Release ` **OR** ` make `
MIGraphX libs can be installed as:
And can be installed by using the 'install' target:
```
make install
```
` cmake --build . --config Release --target install ` **OR** ` make install `
#### Use docker
This will install the library to the `CMAKE_INSTALL_PREFIX` path that was set.
The easiest way to setup the development environment is to use docker. With the dockerfile, you can build a docker image as:
To build a debug version of the library, the cmake variable `CMAKE_BUILD_TYPE` can be set to `Debug`.
docker build -t migraphx .
` cmake -DCMAKE_BUILD_TYPE=Debug . `
Then to enter the developement environment use `docker run`:
docker run --device='/dev/kfd' --device='/dev/dri' -v=`pwd`:/code/AMDMIGraphX -w /code/AMDMIGraphX --group-add video -it migraphx
## Running the tests
In the docker container, all the required prerequisites are already installed, so users can just go to the folder
`/code/AMDMIGraphX` and follow the steps in the above [Build MIGraphX source and install
libs](#building-migraphx-source-and-install-libs)
section to build MIGraphX source.
The tests can be run by using the 'check' target:
### Calling MIGraphX APIs
To use MIGraphX's C/C++ API in your cmake project, we need to set `CMAKE_PREFIX_PATH` to the MIGraphX
installation location and then do
```
find_package(migraphx)
target_link_libraries(myApp migraphx::c)
```
Where `myApp` is the cmake target in your project.
` cmake --build . --config Release --target check ` **OR** ` make check `
## Building the documentation
### Building the documentation
HTML and PDF documentation can be built using:
......@@ -109,13 +192,3 @@ Also, githooks can be installed to format the code per-commit:
```
./.githooks/install
```
## Using docker
The easiest way to setup the development environment is to use docker. You can build the top-level docker file:
docker build -t migraphx .
Then to enter the developement environment use `docker run`:
docker run --device='/dev/kfd' --device='/dev/dri' -v=`pwd`:/data -w /data --group-add video -it migraphx
#!/bin/bash
#
# Build MIGraphX prerequisites for docker container
#install pip3 and rocm-cmake
apt update && apt install -y python3-pip rocm-cmake
# install onnx package for unit tests
pip3 install onnx==1.7.0 numpy==1.18.5 typing==3.7.4 pytest==6.0.1
# install rbuild to build dependencies
pip3 install https://github.com/RadeonOpenCompute/rbuild/archive/master.tar.gz
# rocblas and miopen
apt update && apt install -y rocblas miopen-hip
PREFIX=/usr/local
REQ_FILE_DIR=""
if [ "$#" -ge 2 ]; then
PREFIX=$1
REQ_FILE_DIR=$2
elif [ "$#" -eq 1 ]; then
PREFIX=$1
fi
echo "Dependencies are install at $PREFIX"
# Manually ignore rocm dependencies
cget -p $PREFIX ignore \
RadeonOpenCompute/clang-ocl \
ROCm-Developer-Tools/HIP \
ROCmSoftwarePlatform/MIOpen \
ROCmSoftwarePlatform/MIOpenGEMM \
ROCmSoftwarePlatform/rocBLAS
cget -p $PREFIX init --cxx /opt/rocm/llvm/bin/clang++
cget -p $PREFIX install -f ${REQ_FILE_DIR}dev-requirements.txt
cget -p $PREFIX install oneapi-src/oneDNN@v1.7
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment