Merge branch 'develop' into type-string-driver

doc/src/py_user_guide.rst

+Python User Guide
+=================
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   reference/py

doc/src/reference/cpp.rst

+
+C++ Reference
+=============
+
+shape
+-----
+
+.. doxygenenum:: migraphx_shape_datatype_t
+
+.. doxygenstruct:: migraphx::shape
+
+argument
+--------
+
+.. doxygenstruct:: migraphx::argument
+
+target
+------
+
+.. doxygenstruct:: migraphx::target
+
+program
+-------
+
+.. doxygenstruct:: migraphx::program_parameter_shapes
+
+.. doxygenstruct:: migraphx::program_parameters
+
+.. doxygenstruct:: migraphx_compile_options
+
+.. doxygenstruct:: migraphx::program
+
+quantize
+--------
+
+.. doxygenstruct:: migraphx::quantize_op_names
+
+.. doxygenfunction:: migraphx::quantize_fp16(const program&)
+
+.. doxygenfunction:: migraphx::quantize_fp16(const program&, const quantize_op_names&)
+
+.. doxygenstruct:: migraphx::quantize_int8_options
+
+.. doxygenfunction:: migraphx::quantize_int8
+
+parse_onnx
+----------
+
+.. doxygenstruct:: migraphx::onnx_options
+
+.. doxygenfunction:: migraphx::parse_onnx(const char *)
+
+.. doxygenfunction:: migraphx::parse_onnx(const char *, const migraphx::onnx_options&)
+
+.. doxygenfunction:: migraphx::parse_onnx_buffer(const std::string&)
+
+.. doxygenfunction:: migraphx::parse_onnx_buffer(const std::string&, const migraphx::onnx_options&)
+
+.. doxygenfunction:: migraphx::parse_onnx_buffer(const void *, size_t)
+
+.. doxygenfunction:: migraphx::parse_onnx_buffer(const void *, size_t, const migraphx::onnx_options&)
+
+load
+----
+
+.. doxygenstruct:: migraphx_file_options
+
+.. doxygenfunction:: migraphx::load(const char *)
+
+.. doxygenfunction:: migraphx::load(const char *, migraphx_file_options)
+
+save
+----
+
+.. doxygenfunction:: migraphx::save(const program&, const char *)
+
+.. doxygenfunction:: migraphx::save(const program&, const char *, migraphx_file_options)
+

doc/src/reference/data.rst

-Data types
-==========
-
-shape
------
-
-.. doxygenstruct:: migraphx::shape
-
-literal
--------
-
-.. doxygenstruct:: migraphx::literal
-
-argument
---------
-
-.. doxygenstruct:: migraphx::argument
-
-raw_data
---------
-
-.. doxygenstruct:: migraphx::raw_data
-
-.. doxygenfunction:: migraphx::MIGRAPHX_INLINE_NS::visit_all
-
-
-tensor_view
------------
-
-.. doxygenstruct:: migraphx::tensor_view

doc/src/reference/operators.rst

-Operators
-=========
-
-operation
----------
-
-.. doxygenstruct:: migraphx::operation
-
-operators
----------
-
-.. doxygenfile:: operators.hpp

doc/src/reference/pass.rst

-Passes
-======
-
-pass
-----
-
-.. doxygenstruct:: migraphx::pass
-
-dead_code_elimination
----------------------
-
-.. doxygenstruct:: migraphx::dead_code_elimination
-
-common_subexpression_elimination
---------------------------------
-
-.. doxygenstruct:: migraphx::common_subexpression_elimination
-
-constant_propagate
-------------------
-
-.. doxygenstruct:: migraphx::constant_propagate
-
-eliminate_concat
-----------------
-
-.. doxygenstruct:: migraphx::eliminate_concat
-
-eliminate_contiguous
---------------------
-
-.. doxygenstruct:: migraphx::eliminate_contiguous
-
-fwd_conv_batchnorm_rewrite
---------------------------
-
-.. doxygenstruct:: migraphx::fwd_conv_batchnorm_rewrite
-
-simplify_algebra
-----------------
-
-.. doxygenstruct:: migraphx::simplify_algebra
-
-simplify_reshapes
------------------
-
-.. doxygenstruct:: migraphx::simplify_reshapes

doc/src/reference/program.rst

-Program
-=======
-
-instruction
------------
-
-.. doxygenstruct:: migraphx::instruction
-
-instruction_ref
----------------
-
-.. cpp:type:: migraphx::instruction_ref
-
-    References an instruction in the program.
-
-program
--------
-
-.. doxygenstruct:: migraphx::program
-
-parse_onnx
-----------
-
-.. doxygenfunction:: migraphx::MIGRAPHX_INLINE_NS::parse_onnx

doc/src/reference/py.rst

+.. py:module:: migraphx
+
+Python Reference
+================
+
+shape
+-----
+
+.. py:class:: shape(type, lens, strides=None)
+
+    Describes the shape of a tensor. This includes size, layout, and data type/
+
+.. py:method:: type()
+
+    An integer that represents the type.
+
+    :rtype: int
+
+.. py:method:: lens()
+
+    A list of the lengths of the shape.
+
+    :rtype: list[int]
+
+.. py:method:: strides()
+
+    A list of the strides of the shape.
+
+    :rtype: list[int]
+
+.. py:method:: elements()
+
+    The number of elements in the shape.
+
+    :rtype: int
+
+.. py:method:: bytes()
+
+    The number of bytes the shape uses.
+
+    :rtype: int
+
+.. py:method:: type_size()
+
+    The number of bytes one element uses
+
+    :rtype: int
+
+.. py:method:: packed()
+
+    Returns true if the shape is packed.
+
+    :rtype: bool
+
+.. py:method:: transposed()
+
+    Returns true if the shape is transposed.
+
+    :rtype: bool
+
+.. py:method:: broadcasted()
+
+    Returns true if the shape is broadcasted.
+
+    :rtype: bool
+
+.. py:method:: standard()
+
+    Returns true if the shape is a standard shape. That is, the shape is both packed and not transposed.
+
+    :rtype: bool
+
+.. py:method:: scalar()
+
+    Returns true if all strides are equal to 0 (scalar tensor).
+
+    :rtype: bool
+
+
+argument
+--------
+
+.. py:class:: argument(data)
+
+    Construct an argument from a python buffer. This can include numpy arrays.
+
+.. py:method:: get_shape()
+
+    Returns the shape of the argument.
+
+    :rtype: shape
+
+.. py:method:: tolist()
+
+    Convert the elements of the argument to a python list.
+
+    :rtype: list
+
+
+.. py:function:: generate_argument(s, seed=0)
+
+    Generate an argument with random data.
+
+    :param shape s: Shape of argument to generate.
+    :param int seed: The seed used for random number generation.
+
+    :rtype: argument
+
+.. py:function:: fill_argument(s, value)
+
+    Fill argument of shape s with value.
+
+    :param shape s: Shape of argument to fill.
+    :param int value: Value to fill in the argument.
+
+    :rtype argument
+
+target
+------
+
+.. py:class:: target()
+
+    This represents the compilation target.
+
+.. py:function:: get_target(name)
+
+    Constructs the target.
+
+    :param str name: The name of the target to construct. This can either be 'gpu' or 'ref'.
+
+    :rtype: target
+
+
+module
+------
+.. py:method:: print()
+
+    Prints the contents of the module as list of instructions.
+
+.. py:method:: add_instruction(op, args, mod_args=[])
+    
+    Adds instruction into the module.
+
+    :param operation op: 'migraphx.op' to be added as instruction.
+    :param list[instruction] args: list of inputs to the op.
+    :param list[module] mod_args: optional list of module arguments to the operator.
+    :rtype instruction
+
+.. py:method:: add_literal(data)
+
+    Adds constant or literal data of provided shape into the module from python buffer which includes numpy array.    
+
+    :param py::buffer data: Python buffer or numpy array 
+    :rtype instruction 
+
+.. py:method:: add_parameter(name, shape)
+    
+    Adds a parameter to the module with provided name and shape.
+
+    :param str name: name of the parameter.
+    :param shape shape: shape of the parameter.
+    :rtype instruction
+
+.. py:method:: add_return(args)
+
+    Adds a return instruction into the module.
+
+    :param list[instruction] args: instruction arguments which need to be returned from the module.
+    :rtype instruction
+
+
+program
+-------
+
+.. py:class:: program()
+
+    Represents the computation graph to be compiled and run.
+
+.. py:method:: clone()
+
+    Make a copy of the program.
+
+    :rtype: program
+
+.. py:method:: get_parameter_names()
+ 
+    Get all the input arguments' or parameters' names to the program as a list.
+
+    :rtype list[str]
+
+.. py:method:: get_parameter_shapes()
+
+    Get the shapes of all the input parameters in the program.
+
+    :rtype: dict[str, shape]
+
+.. py:method:: get_output_shapes()
+
+    Get the shapes of the final outputs of the program.
+
+    :rtype: list[shape]
+
+.. py:method:: compile(t, offload_copy=True, fast_math=True)
+
+    Compiles the program for the target and optimizes it.
+
+    :param target t: This is the target to compile the program for.
+    :param bool offload_copy: For targets with offloaded memory(such as the gpu), this will insert instructions during compilation to copy the input parameters to the offloaded memory and to copy the final result from the offloaded memory back to main memory.
+    :param bool fast_math: Optimize math functions to use faster approximate versions. There may be slight accuracy degredation when enabled.
+
+.. py:method:: get_main_module()
+    
+    Get main module of the program.
+
+    :rtype module
+
+.. py:method:: create_module(name)
+    
+    Create and add a module of provided name into the program.
+
+    :param str name : name of the new module.
+    :rtype module
+
+.. py:method:: run(params)
+
+    Run the program.
+
+    :param params: This is a map of the input parameters which will be used when running the program.
+    :type params: dict[str, argument]
+
+    :return: The result of the last instruction.
+    :rtype: list[argument]
+
+.. py:method:: sort()
+
+    Sort the modules of the program such that instructions appear in topologically sorted order.
+
+.. py:function:: quantize_fp16(prog, ins_names=["all"])
+
+    Quantize the program to use fp16.
+
+    :param program prog: Program to quantize.
+    :param ins_names: List of instructions to quantize.
+    :type ins_names: list[str]
+
+
+.. py:function:: quantize_int8(prog, t, calibration=[], ins_names=["dot", "convolution"])
+
+    Quantize the program to use int8.
+
+    :param program prog: Program to quantize.
+    :param target t: Target that will be used to run the calibration data.
+    :param calibration: Calibration data used to decide the parameters to the int8 optimization.
+    :type calibration: list[dict[str, argument]]
+    :param ins_names: List of instructions to quantize.
+    :type ins_names: list[str]
+
+
+op
+--
+.. py::class:: op(name, kwargs)
+
+    Construct an operation with name and arguments.
+    
+    :param str name : name of the operation, must be supported by MIGraphX.
+    :param dict[str, any] kwargs: arguments to the operation.
+    :rtype operation
+
+
+
+parse_onnx
+----------
+
+.. py:function:: parse_onnx(filename, default_dim_value=1, map_input_dims={}, skip_unknown_operators=false, print_program_on_error=false, max_loop_iterations=10)
+
+    Load and parse an onnx file.
+
+    :param str filename: Path to file.
+    :param str default_dim_value: default batch size to use (if not specified in onnx file).
+    :param str map_input_dims: Explicitly specify the dims of an input.
+    :param str skip_unknown_operators: Continue parsing onnx file if an unknown operator is found.
+    :param str print_program_on_error: Print program if an error occurs.
+    :param int max_loop_iterations: Maximum iteration number for the loop operator.
+    :rtype: program
+
+parse_tf
+--------
+
+.. py:function:: parse_tf(filename, is_nhwc=True, batch_size=1, map_input_dims=dict(), output_names=[])
+
+    Load and parse an tensorflow protobuf file file.
+
+    :param str filename: Path to file.
+    :param bool is_nhwc: Use nhwc as default format.
+    :param str batch_size: default batch size to use (if not specified in protobuf).
+    :param dict[str, list[int]] map_input_dims: Optional arg to explictly specify dimensions of the inputs.
+    :param list[str] output_names:  Optional argument specify names of the output nodes.
+    :rtype: program
+
+load
+----
+
+.. py:function:: load(filename, format='msgpack')
+
+    Load a MIGraphX program.
+
+    :param str filename: Path to file.
+    :param str format: Format of file. Valid options are msgpack or json.
+
+    :rtype: program
+
+save
+----
+
+.. py:function:: save(p, filename, format='msgpack')
+
+    Save a MIGraphX program.
+
+    :param program p: Program to save.
+    :param str filename: Path to file.
+    :param str format: Format of file. Valid options are msgpack or json.
+

doc/src/user_guide.rst

-User Guide
-==========
-
-.. toctree::
-   :maxdepth: 2
-   :caption: Contents:
-
-   overview
-   reference/data
-   reference/operators
-   reference/program
-   reference/targets
-   reference/pass

examples/README.md

+# AMD MIGraphX Examples
+
+## Description
+This directory contains examples of common use cases for MIGraphX. 
+
+## Examples:
+- [MIGraphX usage and utilities](./migraphx)
+- [Vision inference examples](./vision)
+- [Natural language inference examples](./nlp)
\ No newline at end of file

examples/migraphx/README.md

+# AMD MIGraphX usage and utilities
+
+- [C++ Parse, Load, and Save Graph Programs](./cpp_parse_load_save)
+- [Exporting Frozen Graphs in TF1](./export_frozen_graph_tf1)
+- [Exporting Frozen Graphs in TF2](./export_frozen_graph_tf2)
+- [MIGraphX Docker Container](./migraphx_docker)
+- [MIGraphX Driver](./migraphx_driver)
\ No newline at end of file

examples/migraphx/cpp_parse_load_save/CMakeLists.txt

+cmake_minimum_required(VERSION 3.5)
+project (PLS)
+
+set (CMAKE_CXX_STANDARD 14)
+set (EXAMPLE parse_load_save)
+
+list (APPEND CMAKE_PREFIX_PATH /opt/rocm/hip /opt/rocm)
+find_package (migraphx)
+
+message("source file: " ${EXAMPLE}.cpp " ---> bin: " ${EXAMPLE})
+add_executable(${EXAMPLE} ${EXAMPLE}.cpp)
+
+target_link_libraries(${EXAMPLE} migraphx::c)

examples/migraphx/cpp_parse_load_save/README.md

+# Parsing, Loading, and Saving MIGraphX Programs
+
+## Description
+This examples demonstrates how to parse, load, and save a graph program using the MIGraphX C++ API. 
+
+## Parsing
+Computation graphs that have been saved in a compatible serialized format, such as [ONNX](https://onnx.ai/get-started.html), can be read in by MIGraphX to create a runable program. 
+
+```
+migraphx::program p;
+unsigned batch = 1; //Or read in as argument
+migraphx::onnx_options options;
+options.set_default_dim_value(batch);
+p = parse_onnx(input_file, options);
+```
+
+## Saving
+An instantiated migraphx::program object can then be serialized to MessagePack (.mxr) format and saved so that it can be loaded for future uses. 
+
+A program can be saved with either of the following:
+```
+migraphx::program p = ... <migraphx::program>;
+migraphx::save(p, output_file); 
+```
+
+```
+migraphx::program p = ... <migraphx::program>;
+migraphx::file_options options;
+options.set_file_format("msgpack");
+migraphx::save(p, output_file, options);
+```
+
+## Loading
+Similarly, graphs that have been previously parsed, and possibly compiled, and then saved in either MessagePack or JSON format can be loaded at later time. 
+
+MessagePack is the default format, and can be loaded with either:
+```
+migraphx::program p;
+p = migraphx::load(input_file);
+```
+
+```
+migraphx::program p;
+migraphx::file_options options;
+options.set_file_format("msgpack");
+p = migraphx::load(input_file, options);
+```
+To load a program that has been saved in JSON format:
+```
+migraphx::program p;
+migraphx::file_options options;
+options.set_file_format("json");
+p = migraphx::load(input_file, options);
+```
+
+
+## Running the Example
+The provided example [`parse_load_save.cpp`](./parse_load_save.cpp) has these features implemented to allow for comparing outputs. 
+
+To compile and run the example from this directory:
+```
+$ mkdir build
+$ cd build
+$ cmake ..
+$ make
+```
+There will now be an executable named `parse_load_save` with the following usage:
+```
+$ ./parse_load_save <input_file> [options]
+options:
+	--parse onnx
+	--load  json/msgpack
+	--save  <output_file>
+```
+
+The program will then attempt to parse or load the graph file, print out its internal graph structure if successful, and optionally save the program to a given file name.

examples/migraphx/cpp_parse_load_save/parse_load_save.cpp

+#include <iostream>
+#include <fstream>
+#include <vector>
+#include <string>
+#include <algorithm>
+
+// MIGraphX C++ API
+#include <migraphx/migraphx.hpp>
+
+char* getCmdOption(char**, char**, const std::string&);
+
+bool cmdOptionExists(char**, char**, const std::string&);
+
+int main(int argc, char** argv)
+{
+    if(argc < 2)
+    {
+        std::cout << "Usage: " << argv[0] << " <input_file> "
+                  << "[options]" << std::endl;
+        std::cout << "options:" << std::endl;
+        std::cout << "\t--parse onnx" << std::endl;
+        std::cout << "\t--load  json/msgpack" << std::endl;
+        std::cout << "\t--save  <output_file>" << std::endl;
+        return 0;
+    }
+
+    char* load_arg         = getCmdOption(argv + 2, argv + argc, "--load");
+    char* save_arg         = getCmdOption(argv + 2, argv + argc, "--save");
+    const char* input_file = argv[1];
+
+    migraphx::program p;
+
+    if(cmdOptionExists(argv + 2, argv + argc, "--parse") ||
+       !cmdOptionExists(argv + 2, argv + argc, "--load"))
+    {
+        std::cout << "Parsing ONNX File" << std::endl;
+        migraphx::onnx_options options;
+        p = parse_onnx(input_file, options);
+    }
+    else if(load_arg != nullptr)
+    {
+        std::cout << "Loading Graph File" << std::endl;
+        std::string format = load_arg;
+        if(format == "json")
+        {
+            migraphx::file_options options;
+            options.set_file_format("json");
+            p = migraphx::load(input_file, options);
+        }
+        else if(format == "msgpack")
+        {
+            migraphx::file_options options;
+            options.set_file_format("msgpack");
+            p = migraphx::load(input_file, options);
+        }
+        else
+            p = migraphx::load(input_file);
+    }
+    else
+    {
+        std::cout << "Error: Incorrect Usage" << std::endl;
+        std::cout << "Usage: " << argv[0] << " <input_file> "
+                  << "[options]" << std::endl;
+        std::cout << "options:" << std::endl;
+        std::cout << "\t--parse onnx" << std::endl;
+        std::cout << "\t--load  json/msgpack" << std::endl;
+        std::cout << "\t--save  <output_file>" << std::endl;
+        return 0;
+    }
+
+    std::cout << "Input Graph: " << std::endl;
+    p.print();
+    std::cout << std::endl;
+
+    if(cmdOptionExists(argv + 2, argv + argc, "--save"))
+    {
+        std::cout << "Saving program..." << std::endl;
+        std::string output_file;
+        output_file = save_arg == nullptr ? "out" : save_arg;
+        output_file.append(".mxr");
+
+        migraphx::file_options options;
+        options.set_file_format("msgpack");
+        migraphx::save(p, output_file.c_str(), options);
+        std::cout << "Program has been saved as ./" << output_file << std::endl;
+    }
+
+    return 0;
+}
+
+char* getCmdOption(char** begin, char** end, const std::string& option)
+{
+    char** itr = std::find(begin, end, option);
+    if(itr != end && ++itr != end)
+    {
+        return *itr;
+    }
+
+    return nullptr;
+}
+
+bool cmdOptionExists(char** begin, char** end, const std::string& option)
+{
+    return std::find(begin, end, option) != end;
+}

examples/migraphx/export_frozen_graph_tf1/README.md

+# Exporting Frozen Graphs in Tensorflow 1
+
+## Description
+This example demonstrates how to export a frozen graph protobuf in Tensorflow 1.X that can be used as input to MIGraphX. Specifically, this is an example of exporting a frozen protobuf of a tensorflow BERT model. 
+
+## How to Use this Example
+
+
+In order to support bert from tensorflow's official [repository](https://github.com/google-research/bert), a serving_input_fn for the estimator must be implemented in [run_classifier.py](https://github.com/google-research/bert/blob/master/run_classifier.py). In this script, insert the following function after importing all libraries and setting up flags:
+
+```
+#...
+flags.DEFINE_integer(
+    "num_tpu_cores", 8,
+    "Only used if `use_tpu` is True. Total number of TPU cores to use.")
+
+# insert function here
+def serving_input_fn():
+    label_ids = tf.placeholder(tf.int32, [None], name='label_ids')
+    input_ids = tf.placeholder(tf.int32, [None, FLAGS.max_seq_length], name='input_ids')
+    input_mask = tf.placeholder(tf.int32, [None, FLAGS.max_seq_length], name='input_mask')
+    segment_ids = tf.placeholder(tf.int32, [None, FLAGS.max_seq_length], name='segment_ids')
+    input_fn = tf.estimator.export.build_raw_serving_input_receiver_fn({
+        'label_ids': label_ids,
+        'input_ids': input_ids,
+        'input_mask': input_mask,
+        'segment_ids': segment_ids,
+    }, default_batch_size=1)()
+    return input_fn
+```
+
+Since we are passing dynamic shape placeholders in the serving_input_fn, the default_batch_size value will essentially determine the resulting shape in the graph.
+
+For inference, we will focus on the "probabilities" layer's output, and we can name this layer by modifying the following [line](https://github.com/google-research/bert/blob/master/run_classifier.py#L608):
+
+```
+probabilities = tf.nn.softmax(logits, axis=-1, name="output")
+
+```
+
+Next, we need to export the saved model after training:
+```
+def main(_):
+# ...
+    with tf.gfile.GFile(output_predict_file, "w") as writer:
+      num_written_lines = 0
+      tf.logging.info("***** Predict results *****")
+      for (i, prediction) in enumerate(result):
+        probabilities = prediction["probabilities"]
+        if i >= num_actual_predict_examples:
+          break
+        output_line = "\t".join(
+            str(class_probability)
+            for class_probability in probabilities) + "\n" 
+        writer.write(output_line)
+        num_written_lines += 1 
+    assert num_written_lines == num_actual_predict_examples
+
+# insert code here
+  if FLAGS.do_train:  # optional to attach export to train flag
+      estimator._export_to_tpu = False
+      estimator.export_savedmodel('saved_models', serving_input_fn)
+# ...
+```
+
+Run bert with the suggested arguments:
+```
+export BERT_BASE_DIR=/path/to/bert/uncased_L-12_H-768_A-12
+export GLUE_DIR=/path/to/glue
+
+python run_classifier.py \
+  --task_name=MRPC \
+  --do_train=true \
+  --do_eval=true \
+  --data_dir=$GLUE_DIR/MRPC \
+  --vocab_file=$BERT_BASE_DIR/vocab.txt \
+  --bert_config_file=$BERT_BASE_DIR/bert_config.json \
+  --init_checkpoint=$BERT_BASE_DIR/bert_model.ckpt \
+  --max_seq_length=128 \
+  --train_batch_size=32 \ # change to appropriate size that fits on GPU
+  --learning_rate=2e-5 \
+  --num_train_epochs=3.0 \
+  --output_dir=/tmp/mrpc_output/
+```
+
+When running, search for the following lines in the output:
+```
+INFO:tensorflow:Restoring parameters from /tmp/model.ckpt-1603
+INFO:tensorflow:SavedModel written to: saved_models/temp-1564086017/saved_model.pb
+```
+
+Note the ID followed by "temp-" (in this case, 1564086017). A directory should exist under saved_models/ that is named with the ID.
+
+We also need to record the name of the output layer in bert. This can be done by inspecting the saved model.
+```
+saved_model_cli show --dir saved_models/1564086017 --tag_set serve --signature_def serving_default
+```
+The output should look like this:
+```
+The given SavedModel SignatureDef contains the following input(s):
+  inputs['input_ids'] tensor_info:
+      dtype: DT_INT32
+      shape: (1, 128)
+      name: input_ids_1:0
+  inputs['input_mask'] tensor_info:
+      dtype: DT_INT32
+      shape: (1, 128)
+      name: input_mask_1:0
+  inputs['label_ids'] tensor_info:
+      dtype: DT_INT32
+      shape: (1)
+      name: label_ids_1:0
+  inputs['segment_ids'] tensor_info:
+      dtype: DT_INT32
+      shape: (1, 128)
+      name: segment_ids_1:0
+The given SavedModel SignatureDef contains the following output(s):
+  outputs['probabilities'] tensor_info:
+      dtype: DT_FLOAT
+      shape: (1, 2)
+      name: loss/output:0
+Method name is: tensorflow/serving/predict
+
+```
+Here the output name is given as "loss/output:0", but we will strip the ":0" from the end, as we are concerned with the node only.
+
+We will use tensorflow's freeze graph utility script and the information gathered above to create the frozen protobuf file. 
+
+```
+CKPT_NUM=1603
+MODEL_ID=1564086017
+OUT_NAME=loss/output
+
+cd /path/to/tensorflow
+python tensorflow/python/tools/freeze_graph.py \
+  --input_graph=/tmp/mrpc_model/graph.pbtxt \
+  --input_binary=false \
+  --input_checkpoint=/tmp/mrpc_model/model.ckpt-${CKPT_NUM} \
+  --input_saved_model_dir=/path/to/bert/saved_models/${MODEL_ID} \
+  --output_graph=/tmp/frozen_bert.pb \
+  --output_node_names=${OUT_NAME}
+```
+
+The final output should be a frozen protobuf that is compatible with MIGraphX.
\ No newline at end of file

examples/migraphx/export_frozen_graph_tf2/.ipynb_checkpoints/Untitled-checkpoint.ipynb

+{
+ "cells": [],
+ "metadata": {},
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

examples/migraphx/export_frozen_graph_tf2/README.md

+# Exporting Frozen Graphs in Tensorflow 2 
+
+## Description
+This example demonstrates how to export a frozen graph protobuf in Tensorflow 2.X that can be used as input for MIGraphX. The method for accomplishing this has changed from Tensorflow 1.X. Please refer to [export_frozen_graphs_tf1]() if you are not yet using Tensorflow 2. 
+
+## How to Use this Example
+If you do not already have Jupyter Notebooks installed, please refer to this [page](https://jupyter.org/install) for instructions. 
+
+Once Jupyter Notebooks is installed, you can navigate to this directory and issue the command:
+
+```
+$ jupyter notebook
+```
+
+From the browser window that is launched, click on `example.ipynb`
+You should now be able to run the notebook from your browser.
+
+To use this on your own models you wish to save, simply edit the first cell to include any additional libraries and modify `MODEL_NAME` and `model` to the model of your choosing. Additionally, training and fine-tuning can be performed before moving on to cells 2 and beyond.
+ 

examples/migraphx/export_frozen_graph_tf2/example.ipynb

+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Exporting Frozen Graphs in Tensorflow 2 \n",
+    "In order to use a trained model as input to MIGraphX, the model must be first be saved in a frozen graph format. This was accomplished in Tensorflow 1 by launching a graph in a tf.Session and then saving the session. However, Tensorflow has decided to deprecate Sessions in favor of functions and SavedModel format.  "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "After importing the necessary libraries, the next step is to instantiate a model. For simplicity, in this example we will use a resnet50 architecture with pre-trained imagenet weights. These weights may also be trained or fine-tuned before freezing. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import tensorflow as tf\n",
+    "tf.enable_eager_execution() #May not be required depending on tensorflow version\n",
+    "from tensorflow.python.framework.convert_to_constants import convert_variables_to_constants_v2\n",
+    "from tensorflow import keras\n",
+    "from tensorflow.keras import layers\n",
+    "\n",
+    "MODEL_NAME = \"resnet50\"\n",
+    "model = tf.keras.applications.ResNet50(weights=\"imagenet\")\n",
+    "model.summary()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## SavedModel format\n",
+    "The simplest way to save a model is through saved\\_model.save()\n",
+    "\n",
+    "This will create an equivalent tensorflow program which can later be loaded for fine-tuning or inference, although it is not directly compatible with MIGraphX."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tf.saved_model.save(model, \"./Saved_Models/{}\".format(MODEL_NAME))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Convert to ConcreteFunction\n",
+    "To begin, we need to get the function equivalent of the model and then concretize the function to avoid retracing."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "full_model = tf.function(lambda x: model(x))\n",
+    "full_model = full_model.get_concrete_function(\n",
+    "    x=tf.TensorSpec(model.inputs[0].shape, model.inputs[0].dtype))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Freeze ConcreteFunction and Serialize\n",
+    "Since we are saving the graph for the purpose of inference, all variables can be made constant (i.e. \"frozen\").\n",
+    "\n",
+    "Next, we need to obtain a serialized GraphDef representation of the graph. \n",
+    "\n",
+    "\n",
+    "Optionally, the operators can be printed out layer by layer followed by the inputs and outputs."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "frozen_func = convert_variables_to_constants_v2(full_model)\n",
+    "frozen_func.graph.as_graph_def()\n",
+    "\n",
+    "layers = [op.name for op in frozen_func.graph.get_operations()]\n",
+    "print(\"-\" * 50)\n",
+    "print(\"Frozen model layers: \")\n",
+    "for layer in layers:\n",
+    "    print(layer)\n",
+    "\n",
+    "print(\"-\" * 50)\n",
+    "print(\"Frozen model inputs: \")\n",
+    "print(frozen_func.inputs)\n",
+    "print(\"Frozen model outputs: \")\n",
+    "print(frozen_func.outputs)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Save Frozen Graph as Protobuf\n",
+    "Finally, we can save to hard drive, and now the frozen graph will be stored as `./frozen_models/<MODEL_NAME>_frozen_graph.pb`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tf.io.write_graph(graph_or_graph_def=frozen_func.graph,\n",
+    "                  logdir=\"./frozen_models\",\n",
+    "                  name=\"{}_frozen_graph.pb\".format(MODEL_NAME),\n",
+    "                  as_text=False)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Assuming MIGraphX has already been built and installed on your system, the driver can be used to verify that the frozen graph has been correctly exported. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import subprocess\n",
+    "driver = \"/opt/rocm/bin/migraphx-driver\"\n",
+    "command = \"read\"\n",
+    "model_path = \"./frozen_models/{}_frozen_graph.pb\".format(MODEL_NAME)\n",
+    "process = subprocess.run([driver, command, model_path], \n",
+    "                         stdout=subprocess.PIPE, \n",
+    "                         universal_newlines=True)\n",
+    "\n",
+    "print(process.stdout)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "tensorflow",
+   "language": "python",
+   "name": "tensorflow"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

examples/migraphx/migraphx_docker/README.md

+# MIGraphX Dockerfile
+
+Instructions for building and running the MIGraphX docker container can be found [here](https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/blob/develop/README.md#using-docker) in this project's top level README.

examples/migraphx/migraphx_driver/README.md

+# MIGraphX Driver
+
+## Description
+The MIGraphX driver is a tool that allows you to utilize many of the core functions of MIGraphX without having to write your own program. 
+
+## How to Use this Example
+
+The MIGraphX driver is installed with MIGraphX and can be found in `/opt/rocm/bin/migraphx-driver`, or in `AMDMIGraphX/build/bin/migraphx-driver` after building the source code. 
+
+See below for a comprehensive list of commands and option arguments, as well as some usage examples.
+
+### Commands
+| Command | Description |
+| --- | ---| 
+| op | When followed by the option --list or -l, prints all operators of MIGraphX |
+| params | Prints the input and output parameter shapes |
+| run | Compiles, allocates parameters, evaluates, and prints input graph |
+| read | Loads and prints input graph |
+| compile | Compiles and prints input graph |
+| verify | Runs reference and GPU implementations and checks outputs for consistency |
+| perf | Compiles and runs input graph then prints performance report |
+
+### Options
+| Option | Description |
+| --- | --- | 
+| --help \| -h | Show help | 
+| --model <resnet50\|inceptionv3\|alexnet> | Loads one of the three default models |
+| --onnx | Load file as onnx graph |
+| --tf | Load file as a tensorflow graph |
+| --migraphx | Load file as a migraphx graph |
+| --migraphx-json | Load file as a migraphx JSON graph |
+| --nhwc | Treat tensorflow format as nhwc | 
+| --nchw | Treat tensorflow format as nchw |
+| --skip-unknown-operators | Skip unknown operators when parsing and continue to parse |
+| --trim \| -t | Trim instructions from the end |
+| --optimize \| -O | Optimize when reading |
+| --graphviz \| -g | Print out a graphviz representation |
+| --brief | Make the output brief |
+| --cpp | Print out the program as cpp program |
+| --json | Print out program as json |
+| --text | Print out program in text format |
+| --binary | Print out program in binary format |
+| --output \| -o | Output to file |
+| --fill0 | Fill parameter with 0s |
+| --fill1 | Fill parameter with 1s |
+| --gpu | Compile on the gpu |
+| --cpu | Compile on the cpu |
+| --ref | Compile on the reference implementation |
+| --enable-offload-copy | Enable implicit offload copying |
+| --disable-fast-math | Disable fast math optimization |
+| --fp16 | Quantize for fp16 |
+| --int8 | Quantize for int8 |
+| --tolerance | Tolerance for errors |
+| --per-instruction \| -i | Verify each instruction |
+| --reduce \| -r | Reduce program and verify |
+| --iterations \| -n | Number of iterations to run for perf report |
+| --list \| -l | List all the operators of MIGraphX |
+
+## Usage Examples
+The examples below supply a simple MNIST ConvNet as the input graph. Models of higher complexity will have considerably larger outputs in most cases.
+
+##### Example: op
+```
+$ /opt/rocm/bin/migraphx-driver op --list
+```
+
+<details>
+<summary>View output</summary>
+
+```
+@literal
+@param
+@return
+abs
+acos
+acosh
+add
+argmax
+argmin
+as_shape
+asin
+asinh
+atan
+atanh
+batch_norm_inference
+broadcast
+capture
+ceil
+check_context::migraphx::version_1::gpu::context
+clip
+concat
+contiguous
+convert
+convolution
+cos
+cosh
+deconvolution
+div
+dot
+elu
+equal
+erf
+exp
+flatten
+floor
+gather
+gpu::abs
+gpu::acos
+gpu::acosh
+gpu::add
+gpu::add_clip
+gpu::add_gelu
+gpu::add_gelu_new
+gpu::add_relu
+gpu::add_tanh
+gpu::argmax
+gpu::argmin
+gpu::asin
+gpu::asinh
+gpu::atan
+gpu::atanh
+gpu::batch_norm_inference
+gpu::ceil
+gpu::clip
+gpu::concat
+gpu::contiguous
+gpu::conv_bias
+gpu::conv_bias_relu
+gpu::convert
+gpu::convolution
+gpu::cos
+gpu::cosh
+gpu::deconv
+gpu::div
+gpu::elu
+gpu::equal
+gpu::erf
+gpu::exp
+gpu::floor
+gpu::gather
+gpu::gelu
+gpu::gelu_new
+gpu::gemm
+gpu::greater
+gpu::int8_conv_pack
+gpu::int8_gemm_pack_a
+gpu::int8_gemm_pack_b
+gpu::layernorm
+gpu::leaky_relu
+gpu::less
+gpu::log
+gpu::logsoftmax
+gpu::lrn
+gpu::max
+gpu::min
+gpu::mul
+gpu::mul_add
+gpu::mul_add_relu
+gpu::pad
+gpu::pooling
+gpu::pow
+gpu::prelu
+gpu::quant_convolution
+gpu::quant_gemm
+gpu::recip
+gpu::record_event
+gpu::reduce_max
+gpu::reduce_mean
+gpu::reduce_min
+gpu::reduce_prod
+gpu::reduce_sum
+gpu::relu
+gpu::rnn_var_sl_last_output
+gpu::rnn_var_sl_shift_output
+gpu::rnn_var_sl_shift_sequence
+gpu::round
+gpu::rsqrt
+gpu::set_stream
+gpu::sigmoid
+gpu::sign
+gpu::sin
+gpu::sinh
+gpu::softmax
+gpu::sqdiff
+gpu::sqrt
+gpu::sub
+gpu::tan
+gpu::tanh
+gpu::triadd
+gpu::triadd_clip
+gpu::triadd_relu
+gpu::triadd_sigmoid
+gpu::triadd_tanh
+gpu::wait_event
+greater
+gru
+hip::allocate
+hip::copy
+hip::copy_from_gpu
+hip::copy_to_gpu
+hip::hip_allocate_memory
+hip::hip_copy_literal
+hip::sync_device
+identity
+im2col
+leaky_relu
+less
+load
+log
+logsoftmax
+lrn
+lstm
+max
+min
+mul
+multibroadcast
+neg
+outline
+pad
+pooling
+pow
+prelu
+quant_convolution
+quant_dot
+recip
+reduce_max
+reduce_mean
+reduce_min
+reduce_prod
+reduce_sum
+ref::batch_norm_inference
+ref::convolution
+ref::deconvolution
+ref::dot
+ref::elu
+ref::im2col
+ref::leaky_relu
+ref::logsoftmax
+ref::lrn
+ref::op
+ref::pad
+ref::pooling_average
+ref::pooling_max
+ref::quant_convolution
+ref::rnn_var_sl_last_output
+ref::softmax
+relu
+reshape
+rnn
+rnn_last_cell_output
+rnn_last_hs_output
+rnn_var_sl_last_output
+rnn_var_sl_shift_output
+rnn_var_sl_shift_sequence
+round
+rsqrt
+scalar
+sigmoid
+sign
+sin
+sinh
+slice
+softmax
+sqdiff
+sqrt
+squeeze
+sub
+tan
+tanh
+transpose
+undefined
+unknown:
+unsqueeze
+```
+
+</details>
+<br/><br/>
+
+##### Example: params
+```
+$ /opt/rocm/bin/migraphx-driver params simple_graph.pb 
+```
+
+<details>
+<summary>View output</summary>
+
+```
+Reading: simple_graph.pb
+x: float_type, {1, 28, 28}, {784, 28, 1}
+```
+
+</details>
+<br/><br/>
+
+##### Example: run (onnx file input)
+```
+$ /opt/rocm/bin/migraphx-driver run --onnx simple_graph.onnx
+```
+
+<details>
+<summary>View output</summary>
+
+```
+Compiling ... 
+Reading: simple_graph.onnx
+@0 = check_context::migraphx::version_1::gpu::context -> float_type, {}, {}
+@1 = hip::hip_allocate_memory[shape=float_type, {256}, {1},id=scratch] -> float_type, {256}, {1}
+@2 = hip::hip_copy_literal[id=@literal:1] -> float_type, {784, 128}, {128, 1}
+x:0 = @param:x:0 -> float_type, {1, 28, 28}, {784, 28, 1}
+@3 = reshape[dims={-1, 784}](x:0) -> float_type, {1, 784}, {784, 1}
+@4 = load[offset=0,end=512](@1) -> float_type, {1, 128}, {128, 1}
+@5 = gpu::gemm[alpha=1,beta=0](@3,@2,@4) -> float_type, {1, 128}, {128, 1}
+@6 = hip::hip_copy_literal[id=@literal:0] -> float_type, {128}, {1}
+@7 = hip::hip_copy_literal[id=@literal:2] -> float_type, {10}, {1}
+@8 = hip::hip_copy_literal[id=@literal:3] -> float_type, {128, 10}, {10, 1}
+@9 = multibroadcast[output_lens={1, 128}](@6) -> float_type, {1, 128}, {0, 1}
+@10 = load[offset=512,end=1024](@1) -> float_type, {1, 128}, {128, 1}
+@11 = gpu::add_relu(@5,@9,@10) -> float_type, {1, 128}, {128, 1}
+@12 = load[offset=0,end=40](@1) -> float_type, {1, 10}, {10, 1}
+@13 = gpu::gemm[alpha=1,beta=0](@11,@8,@12) -> float_type, {1, 10}, {10, 1}
+@14 = multibroadcast[output_lens={1, 10}](@7) -> float_type, {1, 10}, {0, 1}
+@15 = load[offset=40,end=80](@1) -> float_type, {1, 10}, {10, 1}
+@16 = gpu::add(@13,@14,@15) -> float_type, {1, 10}, {10, 1}
+#output_0 = @param:#output_0 -> float_type, {1, 10}, {10, 1}
+@17 = gpu::softmax[axis=1](@16,#output_0) -> float_type, {1, 10}, {10, 1}
+@18 = @return(@17)
+
+Allocating params ... 
+@0 = check_context::migraphx::version_1::gpu::context -> float_type, {}, {}
+@1 = hip::hip_allocate_memory[shape=float_type, {256}, {1},id=scratch] -> float_type, {256}, {1}
+@2 = hip::hip_copy_literal[id=@literal:1] -> float_type, {784, 128}, {128, 1}
+x:0 = @param:x:0 -> float_type, {1, 28, 28}, {784, 28, 1}
+@3 = reshape[dims={-1, 784}](x:0) -> float_type, {1, 784}, {784, 1}
+@4 = load[offset=0,end=512](@1) -> float_type, {1, 128}, {128, 1}
+@5 = gpu::gemm[alpha=1,beta=0](@3,@2,@4) -> float_type, {1, 128}, {128, 1}
+@6 = hip::hip_copy_literal[id=@literal:0] -> float_type, {128}, {1}
+@7 = hip::hip_copy_literal[id=@literal:2] -> float_type, {10}, {1}
+@8 = hip::hip_copy_literal[id=@literal:3] -> float_type, {128, 10}, {10, 1}
+@9 = multibroadcast[output_lens={1, 128}](@6) -> float_type, {1, 128}, {0, 1}
+@10 = load[offset=512,end=1024](@1) -> float_type, {1, 128}, {128, 1}
+@11 = gpu::add_relu(@5,@9,@10) -> float_type, {1, 128}, {128, 1}
+@12 = load[offset=0,end=40](@1) -> float_type, {1, 10}, {10, 1}
+@13 = gpu::gemm[alpha=1,beta=0](@11,@8,@12) -> float_type, {1, 10}, {10, 1}
+@14 = multibroadcast[output_lens={1, 10}](@7) -> float_type, {1, 10}, {0, 1}
+@15 = load[offset=40,end=80](@1) -> float_type, {1, 10}, {10, 1}
+@16 = gpu::add(@13,@14,@15) -> float_type, {1, 10}, {10, 1}
+#output_0 = @param:#output_0 -> float_type, {1, 10}, {10, 1}
+@17 = gpu::softmax[axis=1](@16,#output_0) -> float_type, {1, 10}, {10, 1}
+@18 = @return(@17)
+```
+
+</details>
+<br/><br/>
+
+##### Example: read
+```
+$ /opt/rocm/bin/migraphx-driver read simple_graph.pb 
+```
+
+<details>
+<summary>View output</summary>
+
+```
+Reading: simple_graph.pb
+@0 = @literal{0.0136018, -0.0839988, 0.0375392, 0.0613085, -0.125795, 0.176185, 0.0761055, 0.0093384, -0.110057, -0.170587} -> float_type, {10}, {1}
+@1 = @literal{ ... } -> float_type, {128, 10}, {10, 1}
+@2 = @literal{ ... } -> float_type, {128}, {1}
+@3 = @literal{ ... } -> float_type, {784, 128}, {128, 1}
+@4 = @literal{-1, 784} -> int32_type, {2}, {1}
+x = @param:x -> float_type, {1, 28, 28}, {784, 28, 1}
+@5 = reshape[dims={-1, 784}](x) -> float_type, {1, 784}, {784, 1}
+@6 = identity(@3) -> float_type, {784, 128}, {128, 1}
+@7 = dot[alpha=1,beta=1](@5,@6) -> float_type, {1, 128}, {128, 1}
+@8 = identity(@2) -> float_type, {128}, {1}
+@9 = broadcast[axis=1,dims={1, 128}](@8) -> float_type, {1, 128}, {0, 1}
+@10 = add(@7,@9) -> float_type, {1, 128}, {128, 1}
+@11 = relu(@10) -> float_type, {1, 128}, {128, 1}
+@12 = identity(@1) -> float_type, {128, 10}, {10, 1}
+@13 = dot[alpha=1,beta=1](@11,@12) -> float_type, {1, 10}, {10, 1}
+@14 = identity(@0) -> float_type, {10}, {1}
+@15 = broadcast[axis=1,dims={1, 10}](@14) -> float_type, {1, 10}, {0, 1}
+@16 = add(@13,@15) -> float_type, {1, 10}, {10, 1}
+@17 = softmax[axis=1](@16) -> float_type, {1, 10}, {10, 1}
+@18 = identity(@17) -> float_type, {1, 10}, {10, 1}
+```
+
+</details>
+<br/><br/>
+
+##### Example: compile (on GPU, quantized for fp16)
+```
+$ /opt/rocm/bin/migraphx-driver compile --gpu --fp16 simple_graph.pb
+```
+
+<details>
+<summary>View output</summary>
+
+```
+Compiling ... 
+Reading: simple_graph.pb
+@0 = check_context::migraphx::version_1::gpu::context -> float_type, {}, {}
+@1 = hip::hip_allocate_memory[shape=float_type, {456}, {1},id=scratch] -> float_type, {456}, {1}
+@2 = hip::hip_copy_literal[id=@literal:0] -> half_type, {784, 128}, {128, 1}
+@3 = load[offset=256,end=1824](@1) -> half_type, {1, 28, 28}, {784, 28, 1}
+x = @param:x -> float_type, {1, 28, 28}, {784, 28, 1}
+@4 = gpu::convert[target_type=1](x,@3) -> half_type, {1, 28, 28}, {784, 28, 1}
+@5 = reshape[dims={-1, 784}](@4) -> half_type, {1, 784}, {784, 1}
+@6 = load[offset=0,end=256](@1) -> half_type, {1, 128}, {128, 1}
+@7 = gpu::gemm[alpha=1,beta=0](@5,@2,@6) -> half_type, {1, 128}, {128, 1}
+@8 = hip::hip_copy_literal[id=@literal:2] -> half_type, {128, 10}, {10, 1}
+@9 = hip::hip_copy_literal[id=@literal:1] -> half_type, {128}, {1}
+@10 = hip::hip_copy_literal[id=@literal:3] -> half_type, {10}, {1}
+@11 = load[offset=256,end=512](@1) -> half_type, {1, 128}, {128, 1}
+@12 = broadcast[axis=1,dims={1, 128}](@9) -> half_type, {1, 128}, {0, 1}
+@13 = gpu::add_relu(@7,@12,@11) -> half_type, {1, 128}, {128, 1}
+@14 = load[offset=0,end=20](@1) -> half_type, {1, 10}, {10, 1}
+@15 = gpu::gemm[alpha=1,beta=0](@13,@8,@14) -> half_type, {1, 10}, {10, 1}
+@16 = broadcast[axis=1,dims={1, 10}](@10) -> half_type, {1, 10}, {0, 1}
+@17 = load[offset=20,end=40](@1) -> half_type, {1, 10}, {10, 1}
+@18 = gpu::add(@15,@16,@17) -> half_type, {1, 10}, {10, 1}
+@19 = load[offset=0,end=20](@1) -> half_type, {1, 10}, {10, 1}
+@20 = gpu::softmax[axis=1](@18,@19) -> half_type, {1, 10}, {10, 1}
+output = @param:output -> float_type, {1, 10}, {10, 1}
+@21 = gpu::convert[target_type=2](@20,output) -> float_type, {1, 10}, {10, 1}
+```
+
+</details>
+<br/><br/>
+
+##### Example: verify
+```
+$ /opt/rocm/bin/migraphx-driver verify simple_graph.pb
+```
+
+<details>
+<summary>View output</summary>
+
+```
+Reading: simple_graph.pb
+@0 = @literal{0.0136018, -0.0839988, 0.0375392, 0.0613085, -0.125795, 0.176185, 0.0761055, 0.0093384, -0.110057, -0.170587} -> float_type, {10}, {1}
+@1 = @literal{ ... } -> float_type, {128, 10}, {10, 1}
+@2 = @literal{ ... } -> float_type, {128}, {1}
+@3 = @literal{ ... } -> float_type, {784, 128}, {128, 1}
+@4 = @literal{-1, 784} -> int32_type, {2}, {1}
+x = @param:x -> float_type, {1, 28, 28}, {784, 28, 1}
+@5 = reshape[dims={-1, 784}](x) -> float_type, {1, 784}, {784, 1}
+@6 = identity(@3) -> float_type, {784, 128}, {128, 1}
+@7 = dot[alpha=1,beta=1](@5,@6) -> float_type, {1, 128}, {128, 1}
+@8 = identity(@2) -> float_type, {128}, {1}
+@9 = broadcast[axis=1,dims={1, 128}](@8) -> float_type, {1, 128}, {0, 1}
+@10 = add(@7,@9) -> float_type, {1, 128}, {128, 1}
+@11 = relu(@10) -> float_type, {1, 128}, {128, 1}
+@12 = identity(@1) -> float_type, {128, 10}, {10, 1}
+@13 = dot[alpha=1,beta=1](@11,@12) -> float_type, {1, 10}, {10, 1}
+@14 = identity(@0) -> float_type, {10}, {1}
+@15 = broadcast[axis=1,dims={1, 10}](@14) -> float_type, {1, 10}, {0, 1}
+@16 = add(@13,@15) -> float_type, {1, 10}, {10, 1}
+@17 = softmax[axis=1](@16) -> float_type, {1, 10}, {10, 1}
+@18 = identity(@17) -> float_type, {1, 10}, {10, 1}
+
+@0 = @literal{0.0136018, -0.0839988, 0.0375392, 0.0613085, -0.125795, 0.176185, 0.0761055, 0.0093384, -0.110057, -0.170587} -> float_type, {10}, {1}
+@1 = @literal{ ... } -> float_type, {128, 10}, {10, 1}
+@2 = @literal{ ... } -> float_type, {128}, {1}
+@3 = @literal{ ... } -> float_type, {784, 128}, {128, 1}
+@4 = @literal{-1, 784} -> int32_type, {2}, {1}
+x = @param:x -> float_type, {1, 28, 28}, {784, 28, 1}
+@5 = reshape[dims={-1, 784}](x) -> float_type, {1, 784}, {784, 1}
+@6 = identity(@3) -> float_type, {784, 128}, {128, 1}
+@7 = dot[alpha=1,beta=1](@5,@6) -> float_type, {1, 128}, {128, 1}
+@8 = identity(@2) -> float_type, {128}, {1}
+@9 = broadcast[axis=1,dims={1, 128}](@8) -> float_type, {1, 128}, {0, 1}
+@10 = add(@7,@9) -> float_type, {1, 128}, {128, 1}
+@11 = relu(@10) -> float_type, {1, 128}, {128, 1}
+@12 = identity(@1) -> float_type, {128, 10}, {10, 1}
+@13 = dot[alpha=1,beta=1](@11,@12) -> float_type, {1, 10}, {10, 1}
+@14 = identity(@0) -> float_type, {10}, {1}
+@15 = broadcast[axis=1,dims={1, 10}](@14) -> float_type, {1, 10}, {0, 1}
+@16 = add(@13,@15) -> float_type, {1, 10}, {10, 1}
+@17 = softmax[axis=1](@16) -> float_type, {1, 10}, {10, 1}
+@18 = identity(@17) -> float_type, {1, 10}, {10, 1}
+
+@0 = @literal{0.0136018, -0.0839988, 0.0375392, 0.0613085, -0.125795, 0.176185, 0.0761055, 0.0093384, -0.110057, -0.170587} -> float_type, {10}, {1}
+@1 = @literal{ ... } -> float_type, {128, 10}, {10, 1}
+@2 = @literal{ ... } -> float_type, {128}, {1}
+@3 = @literal{ ... } -> float_type, {784, 128}, {128, 1}
+x = @param:x -> float_type, {1, 28, 28}, {784, 28, 1}
+@4 = ref::reshape[dims={-1, 784}](x) -> float_type, {1, 784}, {784, 1}
+@5 = ref::identity(@3) -> float_type, {784, 128}, {128, 1}
+@6 = ref::dot[alpha=1,beta=1](@4,@5) -> float_type, {1, 128}, {128, 1}
+@7 = ref::identity(@2) -> float_type, {128}, {1}
+@8 = ref::broadcast[axis=1,dims={1, 128}](@7) -> float_type, {1, 128}, {0, 1}
+@9 = ref::contiguous(@8) -> float_type, {1, 128}, {128, 1}
+@10 = ref::add(@6,@9) -> float_type, {1, 128}, {128, 1}
+@11 = ref::relu(@10) -> float_type, {1, 128}, {128, 1}
+@12 = ref::identity(@1) -> float_type, {128, 10}, {10, 1}
+@13 = ref::dot[alpha=1,beta=1](@11,@12) -> float_type, {1, 10}, {10, 1}
+@14 = ref::identity(@0) -> float_type, {10}, {1}
+@15 = ref::broadcast[axis=1,dims={1, 10}](@14) -> float_type, {1, 10}, {0, 1}
+@16 = ref::contiguous(@15) -> float_type, {1, 10}, {10, 1}
+@17 = ref::add(@13,@16) -> float_type, {1, 10}, {10, 1}
+@18 = ref::softmax[axis=1](@17) -> float_type, {1, 10}, {10, 1}
+@19 = ref::identity(@18) -> float_type, {1, 10}, {10, 1}
+
+@0 = check_context::migraphx::version_1::gpu::context -> float_type, {}, {}
+@1 = hip::hip_allocate_memory[shape=float_type, {256}, {1},id=scratch] -> float_type, {256}, {1}
+@2 = hip::hip_copy_literal[id=@literal:3] -> float_type, {784, 128}, {128, 1}
+x = @param:x -> float_type, {1, 28, 28}, {784, 28, 1}
+@3 = load[offset=0,end=512](@1) -> float_type, {1, 128}, {128, 1}
+@4 = reshape[dims={-1, 784}](x) -> float_type, {1, 784}, {784, 1}
+@5 = gpu::gemm[alpha=1,beta=0](@4,@2,@3) -> float_type, {1, 128}, {128, 1}
+@6 = hip::hip_copy_literal[id=@literal:1] -> float_type, {128, 10}, {10, 1}
+@7 = hip::hip_copy_literal[id=@literal:2] -> float_type, {128}, {1}
+@8 = hip::hip_copy_literal[id=@literal:0] -> float_type, {10}, {1}
+@9 = load[offset=512,end=1024](@1) -> float_type, {1, 128}, {128, 1}
+@10 = broadcast[axis=1,dims={1, 128}](@7) -> float_type, {1, 128}, {0, 1}
+@11 = gpu::add_relu(@5,@10,@9) -> float_type, {1, 128}, {128, 1}
+@12 = load[offset=40,end=80](@1) -> float_type, {1, 10}, {10, 1}
+@13 = gpu::gemm[alpha=1,beta=0](@11,@6,@12) -> float_type, {1, 10}, {10, 1}
+@14 = load[offset=0,end=40](@1) -> float_type, {1, 10}, {10, 1}
+@15 = broadcast[axis=1,dims={1, 10}](@8) -> float_type, {1, 10}, {0, 1}
+@16 = gpu::add(@13,@15,@14) -> float_type, {1, 10}, {10, 1}
+output = @param:output -> float_type, {1, 10}, {10, 1}
+@17 = gpu::softmax[axis=1](@16,output) -> float_type, {1, 10}, {10, 1}
+```
+
+</details>
+<br/><br/>
+
+##### Example: perf
+```
+$ /opt/rocm/bin/migraphx-driver perf simple_graph.pb
+```
+
+<details>
+<summary>View output</summary>
+
+```
+Compiling ... 
+Reading: simple_graph.pb
+@0 = check_context::migraphx::version_1::gpu::context -> float_type, {}, {}
+@1 = hip::hip_allocate_memory[shape=float_type, {256}, {1},id=scratch] -> float_type, {256}, {1}
+@2 = hip::hip_copy_literal[id=@literal:3] -> float_type, {784, 128}, {128, 1}
+@3 = load[offset=0,end=512](@1) -> float_type, {1, 128}, {128, 1}
+x = @param:x -> float_type, {1, 28, 28}, {784, 28, 1}
+@4 = reshape[dims={-1, 784}](x) -> float_type, {1, 784}, {784, 1}
+@5 = gpu::gemm[alpha=1,beta=0](@4,@2,@3) -> float_type, {1, 128}, {128, 1}
+@6 = hip::hip_copy_literal[id=@literal:1] -> float_type, {128, 10}, {10, 1}
+@7 = hip::hip_copy_literal[id=@literal:0] -> float_type, {10}, {1}
+@8 = hip::hip_copy_literal[id=@literal:2] -> float_type, {128}, {1}
+@9 = broadcast[axis=1,dims={1, 128}](@8) -> float_type, {1, 128}, {0, 1}
+@10 = load[offset=512,end=1024](@1) -> float_type, {1, 128}, {128, 1}
+@11 = gpu::add_relu(@5,@9,@10) -> float_type, {1, 128}, {128, 1}
+@12 = load[offset=0,end=40](@1) -> float_type, {1, 10}, {10, 1}
+@13 = gpu::gemm[alpha=1,beta=0](@11,@6,@12) -> float_type, {1, 10}, {10, 1}
+@14 = broadcast[axis=1,dims={1, 10}](@7) -> float_type, {1, 10}, {0, 1}
+@15 = load[offset=40,end=80](@1) -> float_type, {1, 10}, {10, 1}
+@16 = gpu::add(@13,@14,@15) -> float_type, {1, 10}, {10, 1}
+output = @param:output -> float_type, {1, 10}, {10, 1}
+@17 = gpu::softmax[axis=1](@16,output) -> float_type, {1, 10}, {10, 1}
+
+Allocating params ... 
+Running performance report ... 
+@0 = check_context::migraphx::version_1::gpu::context -> float_type, {}, {}: 0.00057782ms, 1%
+@1 = hip::hip_allocate_memory[shape=float_type, {256}, {1},id=scratch] -> float_type, {256}, {1}: 0.000295ms, 1%
+@2 = hip::hip_copy_literal[id=@literal:3] -> float_type, {784, 128}, {128, 1}: 0.00027942ms, 1%
+@3 = load[offset=0,end=512](@1) -> float_type, {1, 128}, {128, 1}: 0.000232ms, 1%
+x = @param:x -> float_type, {1, 28, 28}, {784, 28, 1}: 0.0003206ms, 1%
+@4 = reshape[dims={-1, 784}](x) -> float_type, {1, 784}, {784, 1}: 0.00033842ms, 1%
+@5 = gpu::gemm[alpha=1,beta=0](@4,@2,@3) -> float_type, {1, 128}, {128, 1}: 0.212592ms, 52%
+@6 = hip::hip_copy_literal[id=@literal:1] -> float_type, {128, 10}, {10, 1}: 0.00085822ms, 1%
+@7 = hip::hip_copy_literal[id=@literal:0] -> float_type, {10}, {1}: 0.000382ms, 1%
+@8 = hip::hip_copy_literal[id=@literal:2] -> float_type, {128}, {1}: 0.0003486ms, 1%
+@9 = broadcast[axis=1,dims={1, 128}](@8) -> float_type, {1, 128}, {0, 1}: 0.000299ms, 1%
+@10 = load[offset=512,end=1024](@1) -> float_type, {1, 128}, {128, 1}: 0.000234ms, 1%
+@11 = gpu::add_relu(@5,@9,@10) -> float_type, {1, 128}, {128, 1}: 0.0416597ms, 11%
+@12 = load[offset=0,end=40](@1) -> float_type, {1, 10}, {10, 1}: 0.0007548ms, 1%
+@13 = gpu::gemm[alpha=1,beta=0](@11,@6,@12) -> float_type, {1, 10}, {10, 1}: 0.0733071ms, 18%
+@14 = broadcast[axis=1,dims={1, 10}](@7) -> float_type, {1, 10}, {0, 1}: 0.00088142ms, 1%
+@15 = load[offset=40,end=80](@1) -> float_type, {1, 10}, {10, 1}: 0.000408ms, 1%
+@16 = gpu::add(@13,@14,@15) -> float_type, {1, 10}, {10, 1}: 0.0410144ms, 10%
+output = @param:output -> float_type, {1, 10}, {10, 1}: 0.0010222ms, 1%
+@17 = gpu::softmax[axis=1](@16,output) -> float_type, {1, 10}, {10, 1}: 0.0385636ms, 10%
+
+Summary:
+gpu::gemm: 0.285899ms, 69%
+gpu::add_relu: 0.0416597ms, 11%
+gpu::add: 0.0410144ms, 10%
+gpu::softmax: 0.0385636ms, 10%
+hip::hip_copy_literal: 0.00186824ms, 1%
+load: 0.0016288ms, 1%
+@param: 0.0013428ms, 1%
+broadcast: 0.00118042ms, 1%
+check_context::migraphx::version_1::gpu::context: 0.00057782ms, 1%
+reshape: 0.00033842ms, 1%
+hip::hip_allocate_memory: 0.000295ms, 1%
+
+Rate: 2866.1/sec
+Total time: 0.348906ms
+Total instructions time: 0.414369ms
+Overhead time: 0.00348144ms, -0.0654627ms
+Overhead: 1%, -19%
+```
+
+</details>
+<br/><br/>