"magic_pdf/git@developer.sourcefind.cn:wangsen/mineru.git" did not exist on "eb02736a100794784168eebe80f25fe9b85aeda6"
Commit a7de63a4 authored by Chris Austen's avatar Chris Austen
Browse files

Add documentation from 5.1

parent 608c8f96
# Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
config: b8fdf5090e233ad3ae637440d329c855
tags: 645f666f9bcd5a90fca523b33c5a78b7
Contributor Guide
===============
.. toctree::
:maxdepth: 2
:caption: Contents:
dev_intro
dev/data
dev/operators
dev/program
dev/targets
dev/quantization
dev/pass
dev/matchers
dev/tools
C++ User Guide
==============
.. toctree::
:maxdepth: 2
:caption: Contents:
reference/cpp
Data types
==========
shape
-----
.. doxygenstruct:: migraphx::internal::shape
literal
-------
.. doxygenstruct:: migraphx::internal::literal
argument
--------
.. doxygenstruct:: migraphx::internal::argument
raw_data
--------
.. doxygenstruct:: migraphx::internal::raw_data
.. doxygenfunction:: migraphx::internal::visit_all
tensor_view
-----------
.. doxygenstruct:: migraphx::internal::tensor_view
Matchers
========
Introduction
------------
The matchers provide a way compose several predicates together. Many of the matchers can be composed so that ``m(m1, m2)`` will first check that ``m`` matches and then it will check that ``m1`` and ``m2`` will match.
The most commonly-used matcher is the ``name`` matcher. It will match the instruction that have the operator that is equal to the name specified::
auto match_sum = name("sum");
This will find ``sum`` operators. We can also find ``sum`` operators which the output is ``standard_shape``:
auto match_sum = name("sum")(standard_shape());
Arguments
---------
We also want to match arguments to the instructions as well. One way, is to match each argument using the ``arg`` matcher::
auto match_sum = name("sum")(arg(0)(name("@literal"), arg(1)(name("@literal"))));
This will match a ``sum`` operator with the two arguments that are literals. Of course, instead of writing ``arg(0)`` and ``arg(1)`` everytime, the ``args`` matcher can be used::
auto match_sum = name("sum")(args(name("@literal"), name("@literal")));
Binding
-------
As we traverse through the instructions we may want reference some of the instructions we find along the way. We can do this by calling ``.bind``::
auto match_sum = name("sum")(args(
name("@literal").bind("one"),
name("@literal").bind("two")
)).bind("sum");
This will associate the instruction to a name that can be read from the ``matcher_result`` when it matches.
Finding matches
---------------
Finally, when you want to use the matchers to find instructions a callback object can be written which has the matcher and an ``apply`` function which will take the ``matcher_result`` when the match is found::
struct match_find_sum
{
auto matcher() const { return name("sum"); }
void apply(program& p, matcher_result r) const
{
// Do something with the result
}
};
find_matches(prog, match_find_sum{});
Creating matchers
-----------------
There are several ways to create matchers. The macros ``MIGRAPH_BASIC_MATCHER`` and ``MIGRAPH_PRED_MATCHER`` help with creating matchers. For example, we can create a matcher for shapes that are broadcasted::
MIGRAPH_PRED_MATCHER(broadcasted_shape, instruction_ref ins)
{
return ins->get_shape().broadcasted();
}
If we want parameters to the predicate, then we will need to use the ``make_basic_pred_matcher`` to create the matcher. For example, here is how we would create a matcher to check the number of dimensions of the shape::
inline auto number_of_dims(std::size_t n)
{
return make_basic_pred_matcher([=](instruction_ref ins) {
return ins->get_shape().lens().size() == n;
});
}
Operators
=========
operation
---------
.. doxygenstruct:: migraphx::internal::operation
.. doxygenfunction:: migraphx::internal::is_context_free
.. doxygenfunction:: migraphx::internal::has_finalize
operators
---------
.. doxygennamespace:: migraphx::internal::op
Passes
======
pass
----
.. doxygenstruct:: migraphx::internal::pass
dead_code_elimination
---------------------
.. doxygenstruct:: migraphx::internal::dead_code_elimination
eliminate_common_subexpression
------------------------------
.. doxygenstruct:: migraphx::internal::eliminate_common_subexpression
eliminate_concat
----------------
.. doxygenstruct:: migraphx::internal::eliminate_concat
eliminate_contiguous
--------------------
.. doxygenstruct:: migraphx::internal::eliminate_contiguous
eliminate_identity
------------------
.. doxygenstruct:: migraphx::internal::eliminate_identity
eliminate_pad
-------------
.. doxygenstruct:: migraphx::internal::eliminate_pad
propagate_constant
------------------
.. doxygenstruct:: migraphx::internal::propagate_constant
rewrite_batchnorm
-----------------
.. doxygenstruct:: migraphx::internal::rewrite_batchnorm
rewrite_rnn
-----------
.. doxygenstruct:: migraphx::internal::rewrite_rnn
schedule
--------
.. doxygenstruct:: migraphx::internal::schedule
simplify_algebra
----------------
.. doxygenstruct:: migraphx::internal::simplify_algebra
simplify_reshapes
-----------------
.. doxygenstruct:: migraphx::internal::simplify_reshapes
Program
=======
instruction
-----------
.. doxygenstruct:: migraphx::internal::instruction
instruction_ref
---------------
.. cpp:type:: migraphx::internal::instruction_ref
References an instruction in the program.
program
-------
.. doxygenstruct:: migraphx::internal::program
parse_onnx
----------
.. doxygenfunction:: migraphx::internal::parse_onnx
parse_tf
--------
.. doxygenfunction:: migraphx::internal::parse_tf
onnx_options
------------
.. doxygenstruct:: migraphx::internal::onnx_options
tf_options
----------
.. doxygenstruct:: migraphx::internal::tf_options
Quantization
============
quantize_fp16
-------------
.. doxygenfunction:: migraphx::internal::quantize_fp16
quantize_int8
-------------
.. doxygenfunction:: migraphx::internal::quantize_int8
Targets
=======
target
------
.. doxygenstruct:: migraphx::internal::target
gpu::target
-----------
.. doxygenstruct:: migraphx::internal::gpu::target
cpu::target
-----------
.. doxygenstruct:: migraphx::internal::cpu::target
Tools
=====
roctx.py
--------
MIGraphX driver provides `roctx` command which can be used with `rocprof` binary to get marker timing information for each MIGraphX operator.
In order to help user to process timing information, rocTX helper script is provided at `tools/roctx.py`.
The `roctx.py` helper script provides two main functionality: `run` and `parse`. Available knobs and usage are given below:
::
Usage: roctx.py [-h] [--json-path json_path] [--out out]
[--study-name study-name] [--repeat repeat] [--parse]
[--run run] [--debug]
.. option:: --run
Runs `migraphx-driver roctx` command and given `migraphx-driver` knobs, and then parses the results, providing GPU kernel timing information.
MIGraphX knobs can be given via a string to `--run` knob. Please see the examples below.
.. option:: --parse
Given `--json-path`, parses JSON file and provides GPU kernel timing information.
.. option:: --out
Output folder
.. option:: --study-name
Optional. Allows user to name a study for easier interpretation. Defaults to timestamp.
.. option:: --repeat
Number of iterations. Set to **2** by default.
.. option:: --debug
Provides additional debug information related to data. Only use for debugging purposes.
**Examples:**
**Running inference with rocTX for a given ONNX file:**
::
python roctx.py --run '--onnx --gpu fcn-resnet50-11.onnx' --out output_folder --repeat 5
After a run, similar to output given below is expected at terminal. The output will provide `SUM`, `MIN`, `MAX` and `COUNT` information for each kernel executed for a given model.
Average total time is also provided. There are three files provided for reference:
1. `OUTPUT CSV FILE` provides a summary of the run, providing utilized MIGraphX knobs and related kernel timing information
2. `KERNEL TIMING DETAILS` provides the hotspot kernel timing information
3. This will provide all output data related to all iterations executed during a run.
An example output:
.. image:: ./roctx1.jpg
Hotspot kerel timing information:
.. image:: ./roctx2.jpg
**Parsing an already existing JSON file:**
::
python roctx.py --parse --json-path ../trace.json
\ No newline at end of file
MIGraphX Fundamentals
======================
MIGraphX provides an optimized execution engine for deep learning neural networks.
We will cover some simple operations in the MIGraphX framework here.
For a quick start guide to using MIGraphX, look in the examples directory: ``https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/tree/develop/examples/migraphx``.
Location of the Examples
-------------------------
The ``ref_dev_examples.cpp`` can be found in the test directory (``/test``).
The executable file ``test_ref_dev_examples`` based on this file will be created in the ``bin/`` of the build directory after running ``make -j$(nproc) test_ref_dev_examples``.
The executable will also be created when running ``make -j$(nproc) check``, alongside with all the other tests.
Directions for building MIGraphX from source can be found in the main README file: ``https://github.com/ROCmSoftwarePlatform/AMDMIGraphX#readme``.
Adding Two Literals
--------------------
A program is a collection of modules, which are collections of instructions to be executed when calling `eval <migraphx::program::eval>`.
Each instruction has an associated `operation <migraphx::operation>` which represents the computation to be performed by the instruction.
We start with a snippet of the simple ``add_two_literals()`` function::
// create the program and get a pointer to the main module
migraphx::program p;
auto* mm = p.get_main_module();
// add two literals to the program
auto one = mm->add_literal(1);
auto two = mm->add_literal(2);
// make the add operation between the two literals and add it to the program
mm->add_instruction(migraphx::make_op("add"), one, two);
// compile the program on the reference device
p.compile(migraphx::ref::target{});
// evaulate the program and retreive the result
auto result = p.eval({}).back();
std::cout << "add_two_literals: 1 + 2 = " << result << "\n";
We start by creating a simple ``migraphx::program`` object and then getting a pointer to the main module of it.
The program is a collection of ``modules`` that start executing from the main module, so instructions are added to the modules rather than directly onto the program object.
We then use the `add_literal <migraphx::program::add_literal>` function to add an instruction that stores the literal number ``1`` while returning an `instruction_ref <migraphx::instruction_ref>`.
The returned `instruction_ref <migraphx::instruction_ref>` can be used in another instruction as an input.
We use the same `add_literal <migraphx::program::add_literal>` function to add a ``2`` to the program.
After creating the literals, we then create the instruction to add the numbers together.
This is done by using the `add_instruction <migraphx::program::add_instruction>` function with the ``"add"`` `operation <migraphx::program::operation>` created by `make_op <migraphx::program::make_op>` along with the previous `add_literal` `instruction_ref <migraphx::instruction_ref>` for the input arguments of the instruction.
Finally, we can run this `program <migraphx::program>` by compiling it for the reference target (CPU) and then running it with `eval <migraphx::program::eval>`
The result is then retreived and printed to the console.
We can compile the program for the GPU as well, but the file will have to be moved to the ``test/gpu/`` directory and the correct target must be included::
#include <migraphx/gpu/target.hpp>
Using Parameters
-----------------
The previous program will always produce the same value of adding ``1`` and ``2``.
In the next program we want to pass an input to a program and compute a value based on the input.
We can modify the program to take an input parameter ``x``, as seen in the ``add_parameter()`` function::
migraphx::program p;
auto* mm = p.get_main_module();
migraphx::shape s{migraphx::shape::int32_type, {1}};
// add a "x" parameter with the shape s
auto x = mm->add_parameter("x", s);
auto two = mm->add_literal(2);
// add the "add" instruction between the "x" parameter and "two" to the module
mm->add_instruction(migraphx::make_op("add"), x, two);
p.compile(migraphx::ref::target{});
This adds a parameter of type ``int32``, and compiles it for the CPU.
To run the program, we need to pass the parameter as a ``parameter_map`` when we call `eval <migraphx::program::eval>`.
We create the ``parameter_map`` by setting the ``x`` key to an `argument <migraphx::argument>` object with an ``int`` data type::
// create a parameter_map object for passing a value to the "x" parameter
std::vector<int> data = {4};
migraphx::parameter_map params;
params["x"] = migraphx::argument(s, data.data());
auto result = p.eval(params).back();
std::cout << "add_parameters: 4 + 2 = " << result << "\n";
EXPECT(result.at<int>() == 6);
Handling Tensor Data
---------------------
In the previous examples we have only been dealing with scalars, but the `shape <migraphx::shape>` class can describe multi-dimensional tensors.
For example, we can compute a simple convolution::
migraphx::program p;
auto* mm = p.get_main_module();
// create shape objects for the input tensor and weights
migraphx::shape input_shape{migraphx::shape::float_type, {2, 3, 4, 4}};
migraphx::shape weights_shape{migraphx::shape::float_type, {3, 3, 3, 3}};
// create the parameters and add the "convolution" operation to the module
auto input = mm->add_parameter("X", input_shape);
auto weights = mm->add_parameter("W", weights_shape);
mm->add_instruction(migraphx::make_op("convolution", {{"padding", {1, 1}}, {"stride", {2, 2}}}), input, weights);
Here we create two parameters for both the ``input`` and ``weights``.
In the previous examples, we created simple literals, however, most programs will take data from allocated buffers (usually on the GPU).
In this case, we can create `argument <migraphx::argument>` objects directly from the pointers to the buffers::
// Compile the program
p.compile(migraphx::ref::target{});
// Allocated buffers by the user
std::vector<float> a = ...;
std::vector<float> c = ...;
// Solution vector
std::vector<float> sol = ...;
// Create the arguments in a parameter_map
migraphx::parameter_map params;
params["X"] = migraphx::argument(input_shape, a.data());
params["W"] = migraphx::argument(weights_shape, c.data());
// Evaluate and confirm the result
auto result = p.eval(params).back();
std::vector<float> results_vector(64);
result.visit([&](auto output) { results_vector.assign(output.begin(), output.end()); });
EXPECT(migraphx::verify_range(results_vector, sol));
An `argument <migraphx::argument>` can handle memory buffers from either the GPU or the CPU.
By default when running the `program <migraphx::program>`, buffers are allocated on the corresponding target.
When compiling for the CPU, the buffers by default will be allocated on the CPU.
When compiling for the GPU, the buffers by default will be allocated on the GPU.
With the option ``offloaf_copy=true`` set while compiling for the GPU, the buffers will be located on the CPU.
Importing From ONNX
--------------------
A `program <migraphx::program>` can be built directly from an onnx file using the MIGraphX ONNX parser.
This makes it easier to use neural networks directly from other frameworks.
In this case, there is an ``parse_onnx`` function::
program p = migraphx::parse_onnx("model.onnx");
p.compile(migraphx::gpu::target{});
Developer Guide
===============
.. toctree::
:maxdepth: 2
:caption: Contents:
overview
dev/data
dev/operators
dev/program
dev/targets
dev/quantization
dev/pass
dev/matchers
dev/tools
MIGraphX Driver
===============
read
----
.. program:: migraphx-driver read
Loads and prints input graph.
.. include:: ./driver/read.rst
compile
-------
.. program:: migraphx-driver compile
Compiles and prints input graph.
.. include:: ./driver/compile.rst
run
---
.. program:: migraphx-driver run
Loads and prints input graph.
.. include:: ./driver/compile.rst
perf
----
.. program:: migraphx-driver perf
Compiles and runs input graph then prints performance report.
.. include:: ./driver/compile.rst
.. option:: --iterations, -n [unsigned int]
Number of iterations to run for perf report (Default: 100)
verify
------
.. program:: migraphx-driver verify
Runs reference and CPU or GPU implementations and checks outputs for consistency.
.. include:: ./driver/compile.rst
.. option:: --tolerance [double]
Tolerance for errors (Default: 80)
.. option:: -i, --per-instruction
Verify each instruction
.. option:: -r, --reduce
Reduce program and verify
roctx
----
.. program:: migraphx-driver roctx
Provides marker information for each operation, allowing MIGraphX to be used with `rocprof <https://rocmdocs.amd.com/en/latest/ROCm_Tools/ROCm-Tools.html>`_ for performance analysis.
This allows user to get GPU-level kernel timing information.
An example command line combined with rocprof for tracing purposes is given below:
.. code-block:: bash
/opt/rocm/bin/rocprof --hip-trace --roctx-trace --flush-rate 1ms --timestamp on -d <OUTPUT_PATH> --obj-tracking on /opt/rocm/bin/migraphx-driver roctx <ONNX_FILE> <MIGRAPHX_OPTIONS>
After `rocprof` is run, the output directory will contain trace information for HIP, HCC and ROCTX in seperate `.txt` files.
To understand the interactions between API calls, it is recommended to utilize `roctx.py` helper script as desribed in :ref:`dev/tools:rocTX` section.
.. include:: ./driver/compile.rst
\ No newline at end of file
.. include:: ./driver/read.rst
.. option:: --fill0 [std::vector<std::string>]
Fill parameter with 0s
.. option:: --fill1 [std::vector<std::string>]
Fill parameter with 1s
.. option:: --gpu
Compile on the gpu
.. option:: --cpu
Compile on the cpu
.. option:: --ref
Compile on the reference implementation
.. option:: --enable-offload-copy
Enable implicit offload copying
.. option:: --disable-fast-math
Disable fast math optimization
.. option:: --fp16
Quantize for fp16
.. option:: --int8
Quantize for int8
.. option:: <input file>
File to load
.. option:: --model [resnet50|inceptionv3|alexnet]
Load model
.. option:: --onnx
Load as onnx
.. option:: --tf
Load as tensorflow
.. option:: --migraphx
Load as MIGraphX
.. option:: --migraphx-json
Load as MIGraphX JSON
.. option:: --batch [unsigned int] (Default: 1)
Set batch size for model
.. option:: --nhwc
Treat tensorflow format as nhwc
.. option:: --skip-unknown-operators
Skip unknown operators when parsing and continue to parse.
.. option:: --nchw
Treat tensorflow format as nchw
.. option:: --trim, -t [unsigned int]
Trim instructions from the end (Default: 0)
.. option:: --input-dim [std::vector<std::string>]
Dim of a parameter (format: "@name d1 d2 dn")
.. option:: --optimize, -O
Optimize when reading
.. option:: --graphviz, -g
Print out a graphviz representation.
.. option:: --brief
Make the output brief.
.. option:: --cpp
Print out the program as cpp program.
.. option:: --json
Print out program as json.
.. option:: --text
Print out program in text format.
.. option:: --binary
Print out program in binary format.
.. option:: --output, -o [std::string]
Output to file.
.. MIGraphX documentation master file, created by
sphinx-quickstart on Thu Jul 19 11:38:13 2018.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to AMD MIGraphX's documentation!
========================================
.. toctree::
:maxdepth: 3
:caption: Contents:
py_user_guide
cpp_user_guide
driver
contributor_guide
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment