Merge branch 'develop' into uif2-initial

CHANGELOG.md

-# Change Log for MIGraphX
+# Changelog for MIGraphX
 
-Full documentation for MIGraphX is available at [MIGraphX Documentation](https://rocmdocs.amd.com/projects/AMDMIGraphX/en/latest/).
+Full documentation for MIGraphX is available at
+[https://rocmdocs.amd.com/projects/AMDMIGraphX/en/latest/](https://rocmdocs.amd.com/projects/AMDMIGraphX/en/latest/).
 
 ## MIGraphX 2.8 for ROCm 6.0.0
-### Added
-- Support for MI300 GPUs
-- Support for TorchMIGraphX via PyTorch
-- Boosted overall performance by integrating rocMLIR 
-- INT8 support for ONNX Runtime
-- Support for ONNX version 1.14.1
-- Added operators Qlinearadd, QlinearGlobalAveragePool, Qlinearconv, Shrink, CastLike, and RandomUniform operators
-- Added an error message when gpu_targets is not set when compiling migraphx
-- Added parameter to set tolerances with migraphx-driver verify 
-- Added support for MXR files >4 GB 
-- Added MIGRAPHX_TRACE_MLIR flag
-- BETA added capability to use ROCm Composable Kernels via environment variable MIGRAPHX_ENABLE_CK=1
+
+### Additions
+
+* Support for MI300 GPUs
+* Support for TorchMIGraphX via PyTorch
+* Boosted overall performance by integrating rocMLIR
+* INT8 support for ONNX Runtime
+* Support for ONNX version 1.14.1
+* Added new operators: `Qlinearadd`, `QlinearGlobalAveragePool`, `Qlinearconv`, `Shrink`, `CastLike`,
+  and `RandomUniform`
+* Added an error message for when `gpu_targets` is not set during MIGraphX compilation
+* Added parameter to set tolerances with `migraphx-driver` verify
+* Added support for MXR files > 4 GB
+* Added `MIGRAPHX_TRACE_MLIR` flag
+* BETA added capability for using ROCm Composable Kernels via the `MIGRAPHX_ENABLE_CK=1`
+  environment variable
 
 ### Optimizations
-- Improved performance support for INT8
-- Improved time percision while benchmarking candidate kernels from CK or MLIR 
-- Remove contiguous from reshape parsing
-- Updated ConstantOfShape operator to support Dynamic Batch
-- Simplifies dynamic shapes related operators to their static versions if possible
-- Improved debugging tools for accuracy issues
-- Print warning about miopen_fusion while generating mxr 
-- General reduction in system memory usage during model compilation
-- Created additional fusion opportunities during model compilation
-- Improved debugging for matchers
-- Improved general debug messages 
-
-### Fixed
-- Fixed scatter operator for nonstandard shapes with some models from ONNX Model Zoo
-- Provided a compile option to improve accuracy of some models by disabling Fast-Math
-- Improved layernorm + pointwise fusion matching to ignore arguments order
-- Fixed accuracy issue with ROIAlign operator 
-- Fixed Trilu operator computation logic
-- Fixed support for the DETR model 
-
-### Changed
-- Changed migraphx version to 2.8
-- Extracted test packages as its own separate deb file when building migraphx from source
-
-### Removed
-- Removed building Python 2.7 bindings
 
+* Improved performance support for INT8
+* Improved time precision while benchmarking candidate kernels from CK or MLIR
+* Removed contiguous from reshape parsing
+* Updated the `ConstantOfShape` operator to support Dynamic Batch
+* Simplified dynamic shapes-related operators to their static versions, where possible
+* Improved debugging tools for accuracy issues
+* Included a print warning about `miopen_fusion` while generating `mxr`
+* General reduction in system memory usage during model compilation
+* Created additional fusion opportunities during model compilation
+* Improved debugging for matchers
+* Improved general debug messages
+
+### Fixes
+
+* Fixed scatter operator for nonstandard shapes with some models from ONNX Model Zoo
+* Provided a compile option to improve the accuracy of some models by disabling Fast-Math
+* Improved layernorm + pointwise fusion matching to ignore argument order
+* Fixed accuracy issue with `ROIAlign` operator
+* Fixed computation logic for the `Trilu` operator
+* Fixed support for the DETR model
+
+### Changes
+
+* Changed MIGraphX version to 2.8
+* Extracted the test packages into a separate deb file when building MIGraphX from source
+
+### Removals
+
+* Removed building Python 2.7 bindings
 
 ## MIGraphX 2.7 for ROCm 5.7.0
-### Added
-- Enabled hipRTC to not require dev packages for migraphx runtime and allow the ROCm install to be in a different directory than it was during build time
-- Add support for multi-target execution
-- Added Dynamic Batch support with C++/Python APIs
-- Add migraphx.create_argument to python API
-- Added dockerfile example for Ubuntu 22.04
-- Add TensorFlow supported ops in driver similar to exist onnx operator list
-- Add a MIGRAPHX_TRACE_MATCHES_FOR env variable to filter the matcher trace
-- Improved debugging by printing max,min,mean and stddev values for TRACE_EVAL = 2
-- use fast_math flag instead of ENV flag for GELU
-- Print message from driver if offload copy is set for compiled program
+
+### Additions
+
+* hipRTC no longer requires dev packages for MIGraphX runtime and allows the ROCm install to be in a
+   different directory than build time
+* Added support for multi-target execution
+* Added Dynamic Batch support with C++/Python APIs
+* Added `migraphx.create_argument` to Python API
+* Added dockerfile example for Ubuntu 22.04
+* Added TensorFlow supported ops in driver similar to exist onnx operator list
+* Added a MIGRAPHX_TRACE_MATCHES_FOR env variable to filter the matcher trace
+* Improved debugging by printing max,min,mean and stddev values for TRACE_EVAL = 2
+* You can now use the ` fast_math` flag instead of `ENV` for GELU
+* Print message from driver if offload copy is set for compiled program
+
 ### Optimizations
-- Optimized for ONNX Runtime 1.14.0
-- Improved compile times by only building for the GPU on the system
-- Improve performance of pointwise/reduction kernels when using NHWC layouts
-- Load specific version of the migraphx_py library
-- Annotate functions with the block size so the compiler can do a better job of optimizing 
-- Enable reshape on nonstandard shapes
-- Use half HIP APIs to compute max and min
-- Added support for broadcasted scalars to unsqueeze operator
-- Improved multiplies with dot operator
-- Handle broadcasts across dot and concat
-- Add verify namespace for better symbol resolution
-### Fixed
-- Resolved accuracy issues with FP16 resnet50
-- Update cpp generator to handle inf from  float
-- Fix assertion error during verify and make DCE work with tuples
-- Fix convert operation for NaNs
-- Fix shape typo in API test
-- Fix compile warnings for shadowing variable names
-- Add missing specialization for the `nullptr` for the hash function
-### Changed
-- Bumped version of half library to 5.6.0
-- Bumped CI to support rocm 5.6
-- Make building tests optional
-- replace np.bool with bool as per numpy request
-### Removed
-- Removed int8x4 rocBlas calls due to deprecation
-- removed std::reduce usage since not all OS' support it
 
+* Optimized for ONNX Runtime 1.14.0
+* Improved compile times by only building for the GPU on the system
+* Improved performance of pointwise/reduction kernels when using NHWC layouts
+* Loaded specific version of the `migraphx_py` library
+* Annotated functions with the block size so the compiler can do a better job of optimizing
+* Enabled reshape on nonstandard shapes
+* Used half HIP APIs to compute max and min
+* Added support for broadcasted scalars to unsqueeze operator
+* Improved multiplies with dot operator
+* Handled broadcasts across dot and concat
+* Added verify namespace for better symbol resolution
+
+### Fixes
+
+* Resolved accuracy issues with FP16 resnet50
+* Updated cpp generator to handle inf from float
+* Fixed assertion error during verify and made DCE work with tuples
+* Fixed convert operation for NaNs
+* Fixed shape typo in API test
+* Fixed compile warnings for shadowing variable names
+* Added missing specialization for the `nullptr` hash function
+
+### Changees
+
+* Bumped version of half library to 5.6.0
+* Bumped CI to support ROCm 5.6
+* Made building tests optional
+* Replaced `np.bool` with `bool` per NumPy request
+
+### Removals
+
+* Removed int8x4 rocBlas calls due to deprecation
+* Removed `std::reduce` usage because not all operating systems support it
 
 ## MIGraphX 2.5 for ROCm 5.5.0
-### Added
-- Y-Model feature to store tuning information with the optimized model
-- Added Python 3.10 bindings 
-- Accuracy checker tool based on ONNX Runtime
-- ONNX Operators parse_split, and Trilu 
-- Build support for ROCm MLIR
-- Added migraphx-driver flag to print optimizations in python (--python)
-- Added JIT implementation of the Gather and Pad operator which results in better handling of larger tensor sizes.
+
+### Additions
+
+* Y-Model feature will store tuning information with the optimized model
+* Added Python 3.10 bindings
+* Accuracy checker tool based on ONNX runtime
+* ONNX operators parse_split, and Trilu
+* Build support for ROCm MLIR
+* Added the `migraphx-driver` flag to print optimizations in Python (--python)
+* Added JIT implementation of the Gather and Pad operators, which results in better handling for
+  larger tensor sizes
+
 ### Optimizations
-- Improved performance of Transformer based models
-- Improved performance of the Pad, Concat, Gather, and Pointwise operators
-- Improved onnx/pb file loading speed
-- Added general optimize pass which runs several passes such as simplify_reshapes/algebra and DCE in loop.
-### Fixed
-- Improved parsing Tensorflow Protobuf files 
-- Resolved various accuracy issues with some onnx models
-- Resolved a gcc-12 issue with mivisionx
-- Improved support for larger sized models and batches
-- Use --offload-arch instead of --cuda-gpu-arch for the HIP compiler
-- Changes inside JIT to use float accumulator for large reduce ops of half type to avoid overflow.
-- Changes inside JIT to temporarily use cosine to compute sine function.
-### Changed
-- Changed version/location of 3rd party build dependencies to pick up fixes
+
+* Improved performance of Transformer-based models
+* Improved performance of the `Pad`, `Concat`, `Gather`, and `Pointwise` operators
+* Improved ONNX/pb file loading speed
+* Added a general optimize pass that runs several passes, such as `simplify_reshapes`, algebra, and DCE
+  in a loop
+
+### Fixes
+
+* Improved parsing for TensorFlow Protobuf files
+* Resolved various accuracy issues with some ONNX models
+* Resolved a gcc-12 issue with MIVisionX
+* Improved support for larger sized models and batches
+* Use `--offload-arch` instead of `--cuda-gpu-arch` for the HIP compiler
+* Changes inside JIT to use float accumulator for large reduce ops of half type to avoid overflow
+* Changes inside JIT to temporarily use cosine to compute sine function
+
+### Changes
+
+* Changed version and location of third-party build dependencies in order to pick up fixes

Dockerfile

@@ -80,6 +80,10 @@ ADD rbuild.ini /rbuild.ini
 # Temporarily install a new cmake until switching to ubuntu 22.04
 RUN pip3 install cmake==3.22.1
 
+# Location where onnx unit tests models are cached
+ENV ONNX_HOME=/.onnx
+RUN mkdir -p $ONNX_HOME/models && chmod 777 $ONNX_HOME/models
+
 COPY ./tools/install_prereqs.sh /
 RUN /install_prereqs.sh /usr/local / && rm /install_prereqs.sh
 RUN test -f /usr/local/hash || exit 1
@@ -91,11 +95,6 @@ RUN pip3 install yapf==0.28.0
 ADD docs/.sphinx/requirements.txt /doc-requirements.txt
 RUN pip3 install -r /doc-requirements.txt
 
-# Download real models to run onnx unit tests
-ENV ONNX_HOME=/.onnx
-COPY ./tools/download_models.sh /
-RUN /download_models.sh && rm /download_models.sh
-
 # Install latest ccache version
 RUN cget -p $PREFIX install facebook/zstd@v1.4.5 -X subdir -DCMAKE_DIR=build/cmake
 RUN cget -p $PREFIX install ccache@v4.1 -DENABLE_TESTING=OFF

README.md

 # AMD MIGraphX
 
-AMD MIGraphX is AMD's graph inference engine that accelerates machine learning model inference. AMD MIGraphX can be used by
-installing binaries directly or building from source code.
+AMD MIGraphX is AMD's graph inference engine, which accelerates machine learning model inference.
+To use MIGraphX, you can install the binaries or build from source code. Refer to the following sections
+for Ubuntu installation instructions (we'll provide instructions for other Linux distributions in the future).
 
-In the following, instructions of how to build and install MIGraphX are described with Ubuntu as the OS
-(Instructions of installation on other Linux OSes will come later). Note that all the following instructions assume 
-ROCm has been installed successfully. ROCm installation instructions are explained in the [ROCm installation
-guide](https://rocmdocs.amd.com/en/latest/Installation_Guide/Installation-Guide.html).
+```note
+You must [install ROCm](https://rocm.docs.amd.com/en/latest/deploy/linux/quick_start.html) before
+installing MIGraphX.
+```
 
 ## Installing from binaries
-With ROCm installed correctly, MIGraphX binaries can be installed on Ubuntu with the following command:
-```
+
+Install binaries using:
+
+```bash
 sudo apt update && sudo apt install -y migraphx
 ```
-then the header files and libs are installed under `/opt/rocm-<version>`, where `<version>` is the ROCm version.
+
+Header files and libraries are installed under `/opt/rocm-<version>`, where `<version>` is the ROCm
+version.
 
 ## Building from source
 
-There are three ways to build the MIGraphX sources. 
-* [Use the ROCm build tool](#use-the-rocm-build-tool-rbuild)
-    
-    This approach uses [rbuild](https://github.com/RadeonOpenCompute/rbuild) to install the prerequisites and
-build the libs with just one command. 
+You have three options for building from source:
 
-* [Use cmake](#use-cmake-to-build-migraphx)
-    
-    This approach uses a script to install the prerequisites, then use cmake to build the source.
-      
-* [Use docker](#use-docker)
-    
-    This approach builds a docker image with all prerequisites installed, then build the MIGraphX sources inside a docker container. 
+* [ROCm build tool](#use-the-rocm-build-tool-rbuild): Uses
+  [rbuild](https://github.com/RadeonOpenCompute/rbuild) to install prerequisites, then you can build
+  the libraries with a single command.
 
-In the following, we will first list the prerequisites required to build MIGraphX source code, then describe 
-each of the three approaches.
+* [CMake](#use-cmake-to-build-migraphx): Uses a script to install prerequisites, then you can use
+  CMake to build the source.
 
-### List of prerequisites
-The following is a list of prerequisites required to build MIGraphX source. 
+* [Docker](#use-docker): Builds a Docker image with all prerequisites installed, then you can build the
+  MIGraphX sources inside a Docker container.
 
-* [ROCm cmake modules](https://github.com/RadeonOpenCompute/rocm-cmake) **required**
+### Build prerequisites
+
+The following is a list of prerequisites for building MIGraphX.
+
+* [ROCm CMake modules](https://github.com/RadeonOpenCompute/rocm-cmake) **required**
 * [MIOpen](https://github.com/ROCmSoftwarePlatform/MIOpen) for running on the GPU
 * [rocBLAS](https://github.com/ROCmSoftwarePlatform/rocBLAS) for running on the GPU
 * [HIP](https://github.com/ROCm-Developer-Tools/HIP) for running on the GPU
-* [Protobuf](https://github.com/google/protobuf) for reading [onnx](https://github.com/onnx/onnx) files
-* [Half](http://half.sourceforge.net/) - IEEE 754-based half-precision floating point library
-* [pybind11](https://pybind11.readthedocs.io/en/stable/) - for python bindings
-* [JSON](https://github.com/nlohmann/json) - for model serialization to json string format
-* [MessagePack](https://msgpack.org/index.html) - for model serialization to binary format
-* [SQLite3](https://www.sqlite.org/index.html) - to create database of kernels' tuning information or execute queries on existing database
+* [Protobuf](https://github.com/google/protobuf) for reading [onnx](https://github.com/onnx/onnx)
+  files
+* [Half](http://half.sourceforge.net/), an IEEE 754-based half-precision floating point library
+* [pybind11](https://pybind11.readthedocs.io/en/stable/) for python bindings
+* [JSON](https://github.com/nlohmann/json) for model serialization to json string format
+* [MessagePack](https://msgpack.org/index.html) for model serialization to binary format
+* [SQLite3](https://www.sqlite.org/index.html) to create database of kernels' tuning information or run queries on existing database
 
-#### Use the ROCm build tool [rbuild](https://github.com/RadeonOpenCompute/rbuild).
+### Use the ROCm build tool [rbuild](https://github.com/RadeonOpenCompute/rbuild).
 
-In this approach, we use the [rbuild](https://github.com/RadeonOpenCompute/rbuild) build tool to
-build MIGraphX. The specific steps are as follows:
+1. Install `rocm-cmake`, `pip3`, `rocblas`, and `miopen-hip`:
 
-1) Install rocm-cmake, pip3, rocblas, and miopen-hip with the command
+    ```bash
+    sudo apt install -y rocm-cmake python3-pip rocblas miopen-hip
+    ```
 
-```
-sudo apt install -y rocm-cmake python3-pip rocblas miopen-hip
-```
+2. Install [rbuild](https://github.com/RadeonOpenCompute/rbuild) (sudo may be required):
 
-2) Install [rbuild](https://github.com/RadeonOpenCompute/rbuild) (sudo may be required here.)
+    ```bash
+    pip3 install https://github.com/RadeonOpenCompute/rbuild/archive/master.tar.gz
+    ```
 
-```
-pip3 install https://github.com/RadeonOpenCompute/rbuild/archive/master.tar.gz
-```
+3. Build MIGraphX source code:
 
-3) Build MIGraphX source code
+    ```bash
+    rbuild build -d depend -B build
+    ```
 
-```
-rbuild build -d depend -B build
+Once completed, all prerequisites are in the `depend` folder and MIGraphX is in the `build` directory.
+
+```note
+If you get an `rbuild: command not found` error, it's because `rbuild` is installed in `$HOME/.local/bin`,
+which is not in `PATH`. You can either export PATH as `export PATH=$HOME/.local/bin:$PATH` to add
+the folder to `PATH`, or add the option `--prefix /usr/local` in the pip3 command when installing `rbuild`.
 ```
 
-then all the prerequisites are in the folder `depend`, and MIGraphX is built in the `build` directory.
+### Use CMake to build MIGraphX
 
-Also note that you may meet the error of `rbuild: command not found`. It is because rbuild is installed 
-at `$HOME/.local/bin`, which is not in `PATH`. You can either export PATH as `export PATH=$HOME/.local/bin:$PATH` 
-to add the folder to `PATH` or add the option `--prefix /usr/local` in the pip3 command when installing rbuild.
+1. Install the prerequisites:
 
-#### Use cmake to build MIGraphX
+    ```bash
+    rbuild prepare -d depend
+    ```
 
-If using this approach, we need to install the prerequisites, configure the cmake, and then build the source.
+    This puts all the prerequisites are in `depend` the folder. They can be used in the `cmake`
+    configuration as `-DCMAKE_PREFIX_PATH=depend`.
 
-##### Installing the prerequisites
+    If you have sudo access, as an alternative to the `rbuild` command, you can install the prerequisites
+    in the same way as a Dockerfile, by calling `./tools/install_prereqs.sh`.
 
-For convenience, the prerequisites can be built automatically with rbuild as:
+    By default, all prerequisites are installed at the default location (`/usr/local`) and are accessible by all
+    users. For the default location, `sudo` is required to run the script. You can also specify a different
+    location using `./tools/install_prereqs.sh $custom_location`.
 
-```
-rbuild prepare -d depend
-```
+2. Go to the project folder and create a `build` directory:
 
-then all the prerequisites are in the folder `depend`, and they can be used in the `cmake` configuration
-as `-DCMAKE_PREFIX_PATH=depend`.
+    ```bash
+    mkdir build
+    cd build
+    ```
 
-If you have sudo access, as an alternative to the rbuild command, you can install the prerequisites just 
-like in the dockerfile by calling `./tools/install_prereqs.sh`.
+3. Configure CMake. If the prerequisites are installed at the default location `/usr/local`, use:
 
-(Note that this script is for Ubuntu. By default, all prerequisites are installed at the default location `/usr/local` 
-and are accessible by all users. For the default location, `sudo` is required to run the script.
-You can also specify a location at which the prerequisites are installed with `./tools/install_prereqs.sh $your_loc`.)
+    ```bash
+    CXX=/opt/rocm/llvm/bin/clang++ cmake ..
+    ```
 
-##### Building MIGraphX source and install libs
+    Otherwise, you need to set `-DCMAKE_PREFIX_PATH=$your_loc` to configure CMake.
 
-With the above prerequisites installed, we can build source as:
+4. Build MIGraphX source code:
 
-1) Go to the project folder and create a `build` directory:
+    ```cpp
+    make -j$(nproc)
+    ```
 
+    You can verify this using:
 
-```
-mkdir build
-cd build
-```
+    ```cpp
+    make -j$(nproc) check
+    ```
 
-2) Configure the cmake. If the prerequisites are installed at the default location `/usr/local`, the command is:
+5. Install MIGraphX libraries:
 
-```
-CXX=/opt/rocm/llvm/bin/clang++ cmake ..
-```
-Otherwise, you need to set `-DCMAKE_PREFIX_PATH=$your_loc` to configure the cmake. 
+    ```cpp
+    make install
+    ```
 
-3) Build MIGraphX source code
+### Use Docker
 
-```
-make -j$(nproc)
-```
+The easiest way to set up the development environment is to use Docker.
 
-Correctness can be verified as:
+1. With the Dockerfile, build a Docker image:
 
-```
-make -j$(nproc) check
-```
+    ```bash
+        docker build -t migraphx .
+    ```
 
-MIGraphX libs can be installed as:
+2. Enter the development environment using `docker run`:
 
-```
-make install
-```
+    ```bash
+        docker run --device='/dev/kfd' --device='/dev/dri' -v=`pwd`:/code/AMDMIGraphX -w /code/AMDMIGraphX --group-add video -it migraphx
+    ```
 
-#### Use docker
+3. In the Docker container, all required prerequisites are already installed, so you can go to the folder
+    `/code/AMDMIGraphX` and follow the steps (starting from 2) in the
+    [Use CMake to build MIGraphX](#use-cmake-to-build-migraphx).
 
-The easiest way to setup the development environment is to use docker. With the dockerfile, you can build a docker image as:
+## Using the MIGraphX Python module
 
-    docker build -t migraphx .
+To use MIGraphX's Python module, you can set `PYTHONPATH` or use the `.deb` package:
 
-Then to enter the developement environment use `docker run`:
+* Setting `PYTHONPATH`:
 
-    docker run --device='/dev/kfd' --device='/dev/dri' -v=`pwd`:/code/AMDMIGraphX -w /code/AMDMIGraphX --group-add video -it migraphx
+    ```bash
+    export PYTHONPATH=/opt/rocm/lib:$PYTHONPATH
+    ```
 
-In the docker container, all the required prerequisites are already installed, so users can just go to the folder 
-`/code/AMDMIGraphX` and follow the steps in the above [Build MIGraphX source and install
-libs](#building-migraphx-source-and-install-libs)
-section to build MIGraphX source.
+* Creating the `deb` package:
 
-### Using MIGraphX Python Module
-To use MIGraphX's Python module, please either set `PYTHONPATH` or use `.deb` package as explained below:
+    ```bash
+    make package
+    ```
 
-- Setting `PYTHONPATH` :
-```
-export PYTHONPATH=/opt/rocm/lib:$PYTHONPATH
-```
-- Creating and installing the package:
+    This provides the path for .deb package.
 
-To create deb package:
-```
-make package
-```
-This will provide the path of .deb package.
+    To install:
 
-To install:
-```
-dpkg -i <path_to_deb_file>
-```
+    ```bash
+    dpkg -i <path_to_deb_file>
+    ```
 
-### Calling MIGraphX APIs
-To use MIGraphX's C/C++ API in your cmake project, we need to set `CMAKE_PREFIX_PATH` to the MIGraphX
-installation location and then do 
-```
+## Calling MIGraphX APIs
+
+To use MIGraphX's C/C++ API in your CMake project, you must set `CMAKE_PREFIX_PATH` to the
+MIGraphX installation location and run:
+
+```bash
 find_package(migraphx)
 target_link_libraries(myApp migraphx::c)
 ```
-Where `myApp` is the cmake target in your project.
+
+Where `myApp` is the CMake target in your project.
 
 ## Building for development
 
-Using rbuild, the dependencies for development can be installed with:
+Using `rbuild`, you can install the dependencies for development with:
 
-```
+```bash
 rbuild develop
 ```
 
-This will install the dependencies for development into the `deps` directory and
-configure `cmake` to use those dependencies in the `build` directory. These
-directories can be changed by passing the `--deps-dir` and `--build-dir` flags
-to `rbuild` command:
+This installs development dependencies in the `deps` directory and configures `cmake` to use those
+dependencies in the `build` directory. You can change these directories by passing the `--deps-dir` and
+`--build-dir` flags to the `rbuild` command:
 
-```
+```bash
 rbuild develop --build-dir build_rocm_55 --deps-dir /home/user/deps_dir
 ```
 
@@ -223,12 +227,12 @@ Depending on your setup `sudo` may be required for the pip install.
 
 All the code is formatted using clang-format. To format a file, use:
 
-```
+```clang
 clang-format-10 -style=file -i <path-to-source-file>
 ```
 
 Also, githooks can be installed to format the code per-commit:
 
-```
+```bash
 ./.githooks/install
 ```

src/driver/main.cpp

@@ -761,7 +761,8 @@ struct main_command
     {
         std::string version_str = "MIGraphX Version: " + std::to_string(MIGRAPHX_VERSION_MAJOR) +
                                   "." + std::to_string(MIGRAPHX_VERSION_MINOR) + "." +
-                                  std::to_string(MIGRAPHX_VERSION_PATCH) + "." MIGRAPHX_VERSION_TWEAK;
+                                  std::to_string(MIGRAPHX_VERSION_PATCH) +
+                                  "." MIGRAPHX_VERSION_TWEAK;
         ap(wrong_commands, {}, ap.metavar("<command>"), ap.append());
         ap(nullptr, {"-h", "--help"}, ap.help("Show help"), ap.show_help(get_command_help()));
         ap(nullptr,

src/include/migraphx/op/multinomial.hpp

 /*
  * The MIT License (MIT)
  *
- * Copyright (c) 2015-2022 Advanced Micro Devices, Inc. All rights reserved.
+ * Copyright (c) 2015-2023 Advanced Micro Devices, Inc. All rights reserved.
  *
  * Permission is hereby granted, free of charge, to any person obtaining a copy
  * of this software and associated documentation files (the "Software"), to deal
@@ -21,11 +21,52 @@
  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  * THE SOFTWARE.
  */
+
+/**
+ *  * Multinomial or categorical distribution.  Performs a sampling of random input
+ *         and returns a count of
+ *         each category, or bucket.  This does not require the standard multinomial
+ *         distribution but instead takes a probability distribution, i.e. cumulative
+ *         distribution function (CDF) as its first input.
+ *
+ *      Inputs:   args[0] - a tensor of probabilities for each category.  Values are
+ *                          cumulative density function
+ *                          totals as provided by operation prefix_scan_sum.  Values are
+ *                          cumulative probabilities (i.e. start with any set of numbers > 0
+ *                          and then apply prefix_scan_sum).  Values do not need to be
+ *                          normalized to sum to 1; this is done in runtime computation.
+ *
+ *                          This input has Rank 2.  Dimension 0 is batch #, so that there can be
+ *                          a different CDF for each iteration in the batch.  The size of dimension
+ *                          1 is the number of categories.
+ *
+ *                args[1] - a tensor of random numbers.  The last dimension is the sample
+ *                          size, i.e. the number of
+ *                          random samples in each iteration of the batch.  Nominally
+ *                          has two dimensions where the first dimension is batch size, but
+ *                          any reshaping such that the total
+ *                          number of elements is (batch_size * sample_size) is legal.
+ *
+ *                          Values as created by a std::mt19937 like this:
+ *
+ *                           size_t sample_size = 100000;
+ *                           float seed         = 0.0f;
+ *                           std::mt19937 gen(seed);
+ *                           std::uniform_real_distribution<> dis(0.0, 1.0);
+ *                           std::vector<float> rand_samples(sample_size);
+ *                           std::generate(rand_samples.begin(), rand_samples.end(), [&]() { return
+ *                                dis(gen); });
+ *
+ *        Output:   A 2D vector of category each input.  Dimensions are (Input 1[first], Input
+ 2[last]).
+ *
+*/
 #ifndef MIGRAPHX_GUARD_OPERATORS_MULTINOMIAL_HPP
 #define MIGRAPHX_GUARD_OPERATORS_MULTINOMIAL_HPP
 
-#include <migraphx/check_shapes.hpp>
 #include <migraphx/argument.hpp>
+#include <migraphx/check_shapes.hpp>
+#include <migraphx/dyn_output.hpp>
 #include <migraphx/par_for.hpp>
 #include <migraphx/reflect.hpp>
 #include <random>
@@ -47,22 +88,35 @@ struct multinomial
     std::string name() const { return "multinomial"; }
     shape compute_shape(std::vector<shape> inputs) const
     {
-        check_shapes{inputs, *this}.has(2).only_dims(2);
-        size_t sample_size = inputs.back().lens().back();
+        check_shapes{inputs, *this, true}.has(2).only_dims(2);
 
-        if(not contains({shape::int32_type, shape::int64_type}, dtype))
-            MIGRAPHX_THROW(
-                "Multinomial: Invalid output type. Valid types are int32_type and int64_type.");
+        if(inputs.back().ndim() < 1)
+            MIGRAPHX_THROW("Multinomial: Second input shape (sample) has no dimensions");
+        if(dtype == shape::bool_type)
+            MIGRAPHX_THROW("Multinomial: boolean output type invalid.");
 
-        return {dtype, {inputs.front().lens().front(), sample_size}};
+        // Output takes one dimension from each of the two input shapes.  If they are both fixed,
+        // return a static shape
+        if((not inputs.front().dynamic()) or (inputs.front().dyn_dims().front().is_fixed()))
+        {
+            if((not inputs.back().dynamic()) or (inputs.back().dyn_dims().back().is_fixed()))
+            {
+                size_t batch = {inputs.front().max_lens().front()};
+                size_t sample_size{inputs.back().max_lens().back()};
+                return {dtype, {batch, sample_size}};
+            }
+        }
+        return {dtype,
+                {inputs.front().to_dynamic().dyn_dims().front(),
+                 inputs.back().to_dynamic().dyn_dims().back()}};
     }
 
-    argument compute(const shape& output_shape, std::vector<argument> args) const
+    argument compute(const dyn_output& dyn_out, std::vector<argument> args) const
     {
-        argument result{output_shape};
-        size_t batch_size  = output_shape.lens().front();
+        argument result{dyn_out.computed_shape};
+        size_t batch_size  = dyn_out.computed_shape.lens().front();
         size_t class_size  = args[0].get_shape().lens().back();
-        size_t sample_size = output_shape.lens().back();
+        size_t sample_size = dyn_out.computed_shape.lens().back();
 
         visit_all(args[0], args[1])([&](auto cdf, auto dist) {
             result.visit([&](auto output) {
@@ -70,13 +124,16 @@ struct multinomial
                     auto idx       = args[1].get_shape().multi(i);
                     auto cdf_begin = cdf.begin() + (idx[0] * class_size);
                     auto cdf_end   = cdf_begin + class_size;
+
+                    // std::upper_bound returns an iterator to the bucket the value belongs in,
+                    // when normalized by the probability distribution dist
                     auto sample_iter =
                         std::upper_bound(cdf_begin, cdf_end, dist[i] * *(std::prev(cdf_end)));
+                    // convert iterator to an integer index
                     output[i] = std::distance(cdf_begin, sample_iter);
                 });
             });
         });
-
         return result;
     }
 };

src/include/migraphx/op/prefix_scan_op.hpp

@@ -22,6 +22,12 @@
  * THE SOFTWARE.
  */
 
+/**
+ * Parent struct for prefix scan ops.  A prefix scan is a mathematical entity useful
+ * in parallelizing various computations.  Given a list of numbers, a prefix scan
+ * op returns an equal size list of running totals of the values.  Other operations
+ * besides addition can be supported by child ops.
+ */
 #ifndef MIGRAPHX_GUARD_OPERATORS_SCAN_OP_HPP
 #define MIGRAPHX_GUARD_OPERATORS_SCAN_OP_HPP
 

src/include/migraphx/op/random_uniform.hpp

@@ -65,11 +65,10 @@ struct random_uniform
         return inputs.at(1);
     }
 
-    argument compute(const shape&, std::vector<argument> args) const
+    argument compute(const dyn_output& dyn_out, std::vector<argument> args) const
     {
         // Output goes into the passed buffer, not the shape output.
-        auto result = args[1];
-
+        argument result{dyn_out.computed_shape};
         uint64_t local_seed = args[0].at<uint64_t>(0);
         std::mt19937 gen(local_seed);
 

src/onnx/parse_clip.cpp

 /*
  * The MIT License (MIT)
  *
- * Copyright (c) 2015-2022 Advanced Micro Devices, Inc. All rights reserved.
+ * Copyright (c) 2015-2023 Advanced Micro Devices, Inc. All rights reserved.
  *
  * Permission is hereby granted, free of charge, to any person obtaining a copy
  * of this software and associated documentation files (the "Software"), to deal

src/onnx/parse_multinomial.cpp

 /*
  * The MIT License (MIT)
  *
- * Copyright (c) 2015-2022 Advanced Micro Devices, Inc. All rights reserved.
+ * Copyright (c) 2015-2023 Advanced Micro Devices, Inc. All rights reserved.
  *
  * Permission is hereby granted, free of charge, to any person obtaining a copy
  * of this software and associated documentation files (the "Software"), to deal
@@ -41,6 +41,9 @@ struct parse_multinomial : op_parser<parse_multinomial>
                           const onnx_parser::node_info& info,
                           std::vector<instruction_ref> args) const
     {
+        if(args.empty())
+            MIGRAPHX_THROW("PARSE_MULTINOMIAL: no arguments given");
+
         int dtype = 6;
         if(contains(info.attributes, "dtype"))
             dtype = info.attributes.at("dtype").i();
@@ -49,35 +52,90 @@ struct parse_multinomial : op_parser<parse_multinomial>
         size_t sample_size = 1;
         if(contains(info.attributes, "sample_size"))
             sample_size = info.attributes.at("sample_size").i();
+        else
+            MIGRAPHX_THROW("PARSE_MULTINOMIAL: sample_size not given");
+
+        // Use logarithmic math to scale probabilities while avoiding division by very
+        // small numbers.  Scaling by the maximum makes very tiny ranges more
+        // tractable; any constant factor gives equivalent distr. since the Multinomial op.
+        // normalizes at runtime.
 
         // Subtract the per-batch maximum log-probability, making the per-batch max 0
         auto maxes =
             info.add_instruction(migraphx::make_op("reduce_max", {{"axes", {1}}}), args[0]);
-        auto mb_maxes = info.add_instruction(
-            migraphx::make_op("multibroadcast", {{"out_lens", args[0]->get_shape().lens()}}),
-            maxes);
-        auto cdf = info.add_instruction(migraphx::make_op("sub"), args[0], mb_maxes);
+        auto cdf = info.add_common_op("sub", args[0], maxes);
         // Take the element-wise exponent to get probabilities in the range (0, 1]
         cdf = info.add_instruction(migraphx::make_op("exp"), cdf);
-        // Compute the cumulative density function
+        // Compute the cumulative distribution function
         cdf = info.add_instruction(
             migraphx::make_op("prefix_scan_sum", {{"axis", 1}, {"exclusive", false}}), cdf);
 
-        // Pre-compute random distribution
-        std::mt19937 gen(std::chrono::high_resolution_clock::now().time_since_epoch().count());
+        instruction_ref seed_input;
         if(contains(info.attributes, "seed"))
-            gen.seed(info.attributes.at("seed").f());
+        {
+            float seed = info.attributes.at("seed").f();
+            migraphx::shape s{migraphx::shape::float_type, {1}};
+            std::vector<float> data = {seed};
+            seed_input              = info.add_literal(migraphx::literal(s, data));
+        }
+        else
+        {
+            seed_input = info.add_instruction(migraphx::make_op("random_seed"));
+        }
+        instruction_ref randoms;
+
+        shape s0 = args[0]->get_shape();
+
+        if(s0.dynamic())
+        {
+            //  Dynamic batch_size will be taken from args[0].  The input argument to this should
+            // have a second dimension of sample_size.
+            std::vector<shape::dynamic_dimension> dyn_dim_set;
+            dyn_dim_set.emplace_back(s0.dyn_dims().front());
+            dyn_dim_set.emplace_back(shape::dynamic_dimension{sample_size, sample_size});
+
+            // read the input dimensions
+            auto dim_of =
+                info.add_instruction(migraphx::make_op("dimensions_of", {{"end", 2}}), args[0]);
+
+            // The next two operations insert the value sample_size into the second array position
+
+            // make an argument of (1, 0)
+            shape s(shape::int64_type, {2});
+            std::vector<int64_t> data1{1, 0};
+            auto l1        = info.add_literal(s, data1);
+            auto batch_arg = info.add_instruction(migraphx::make_op("mul"), dim_of, l1);
+            std::vector<int64_t> data2(2, 0);
+            // make an argument of (0, sample_size)
+            data2[1]         = sample_size;
+            auto l2          = info.add_literal(s, data2);
+            auto alloc_shape = info.add_instruction(migraphx::make_op("add"), batch_arg, l2);
+            // alloc_shape should contain the input-based shape dimensions as its values at runtime,
+            // and its own shape is {2}
+
+            // compile_shape is the shape used when compiling the Allocate op, and may be dynamic
+            migraphx::shape compile_shape =
+                migraphx::shape(s0.type(), {s0.dyn_dims().front(), {sample_size, sample_size}});
 
-        std::uniform_real_distribution<> dis(0.0, 1.0);
-        size_t batch_size = args[0]->get_shape().lens().front();
-        migraphx::shape dist_shape{migraphx::shape::float_type, {batch_size, sample_size}};
+            // Allocate on-device storage for the random values
+            auto alloc = info.add_instruction(
+                migraphx::make_op("allocate", {{"shape", to_value(compile_shape)}}), alloc_shape);
+            randoms = info.add_instruction(migraphx::make_op("random_uniform"), seed_input, alloc);
+        }
+        else
+        {
+            // use literal.  The array populated by random_uniform may have any shape, as long its
+            // number of elements is batch_size * sample_size .
+            size_t batch_size = s0.lens().front();
+            auto rand_dummy   = info.add_literal(
+                migraphx::literal{migraphx::shape::float_type, {batch_size * sample_size}});
 
-        std::vector<float> random_dist(batch_size * sample_size);
-        std::generate(random_dist.begin(), random_dist.end(), [&]() { return dis(gen); });
-        auto dist_lit = info.add_literal(migraphx::literal{dist_shape, random_dist});
+            randoms =
+                info.add_instruction(migraphx::make_op("random_uniform"), seed_input, rand_dummy);
+        }
 
         return info.add_instruction(
-            migraphx::make_op("multinomial", {{"dtype", output_type}}), cdf, dist_lit);
+            migraphx::make_op("multinomial", {{"dtype", output_type}}), cdf, randoms);
     }
 };
 

src/onnx/parse_split.cpp

@@ -68,13 +68,34 @@ struct parse_split : op_parser<parse_split>
         // no split attribute, input is equally divided
         else
         {
-            if((lens[tuned_axis] % info.num_outputs) != 0)
+            std::size_t num_outputs = info.num_outputs;
+            // the num_outputs attribute seems to be redundant since we already have
+            // node_info::num_outputs, but we can still perform an error check
+            if(contains(info.attributes, "num_outputs"))
             {
-                MIGRAPHX_THROW("PARSE_SPLIT: input cannot be equally divided into " +
-                               std::to_string(info.num_outputs) + " splits!");
+                num_outputs =
+                    parser.parse_value(info.attributes.at("num_outputs")).at<std::size_t>();
+                if(num_outputs != info.num_outputs)
+                {
+                    MIGRAPHX_THROW("PARSE_SPLIT: num_outputs attribute " +
+                                   std::to_string(num_outputs) +
+                                   " doesn't match actual number of outputs " +
+                                   std::to_string(info.num_outputs) + "!");
+                }
+            }
+
+            if(lens[tuned_axis] % num_outputs == 0)
+            {
+                std::size_t chunk_size = lens[tuned_axis] / num_outputs;
+                vec_splits.resize(num_outputs, chunk_size);
+            }
+            else
+            {
+                std::size_t chunk_size      = lens[tuned_axis] / num_outputs + 1;
+                std::size_t last_chunk_size = lens[tuned_axis] - chunk_size * (num_outputs - 1);
+                vec_splits.resize(num_outputs - 1, chunk_size);
+                vec_splits.push_back(last_chunk_size);
             }
-            auto dl = lens[tuned_axis] / info.num_outputs;
-            vec_splits.resize(info.num_outputs, dl);
         }
 
         if(std::accumulate(vec_splits.begin(), vec_splits.end(), int64_t(0)) !=

test/api/CMakeLists.txt

@@ -30,6 +30,9 @@ function(add_api_test TEST_NAME TEST_SRC TEST_DIR)
     add_test(NAME ${NAME} COMMAND $<TARGET_FILE:${NAME}> WORKING_DIRECTORY ${TEST_DIR}) 
     add_dependencies(tests ${NAME})
     add_dependencies(check ${NAME})
+    if(WIN32)
+        target_compile_definitions(${NAME} PRIVATE _CRT_SECURE_NO_WARNINGS)
+    endif()
 endfunction()
 
 # Workaround: C file dont work with clang-tidy right now, need a fix in rocm-cmake
@@ -41,6 +44,9 @@ function(add_c_api_test TEST_NAME TEST_SRC TEST_DIR)
     add_test(NAME ${NAME} COMMAND $<TARGET_FILE:${NAME}> WORKING_DIRECTORY ${TEST_DIR}) 
     add_dependencies(tests ${NAME})
     add_dependencies(check ${NAME})
+    if(WIN32)
+        target_compile_definitions(${NAME} PRIVATE _CRT_SECURE_NO_WARNINGS)
+    endif()
 endfunction()
 
 add_api_test(array_base test_array_base.cpp ${TEST_ONNX_DIR})

test/onnx/gen_onnx.py

@@ -4883,9 +4883,9 @@ def mod_test_fmod_different_dtypes():
 
 @onnx_test()
 def multinomial_test():
-    sample_size = 10
-    seed = 0.0
-    input = helper.make_tensor_value_info("input", TensorProto.FLOAT, [1, 10])
+    sample_size = 13
+    seed = 0.
+    input = helper.make_tensor_value_info("input", TensorProto.FLOAT, [3, 10])
     output = helper.make_tensor_value_info("output", TensorProto.INT32,
                                            [1, 10])
 
@@ -4898,6 +4898,44 @@ def multinomial_test():
     return ([node], [input], [output])
 
 
+@onnx_test()
+def multinomial_dyn_test():
+    sample_size = 100000
+    seed = 1.3
+    categories = 5
+    input = helper.make_tensor_value_info("input", TensorProto.FLOAT,
+                                          [None, categories])
+    output = helper.make_tensor_value_info("output", TensorProto.FLOAT,
+                                           [None, categories])
+
+    node = onnx.helper.make_node(
+        'Multinomial',
+        inputs=['input'],
+        sample_size=sample_size,
+        dtype=1,  # shape::float_type
+        seed=seed,
+        outputs=['output'])
+
+    return ([node], [input], [output])
+
+
+@onnx_test()
+def multinomial_autoseed_dyn_test():
+    # If seed attribute is not given, device should auto generate one at runtime
+    sample_size = 12
+    input = helper.make_tensor_value_info("input", TensorProto.FLOAT,
+                                          [None, 10])
+    output = helper.make_tensor_value_info("output", TensorProto.INT32,
+                                           [None, 10])
+
+    node = onnx.helper.make_node('Multinomial',
+                                 inputs=['input'],
+                                 sample_size=sample_size,
+                                 outputs=['output'])
+
+    return ([node], [input], [output])
+
+
 @onnx_test()
 def multinomial_generated_seed_test():
     sample_size = 10
@@ -8042,6 +8080,42 @@ def split_test_no_attribute():
     return ([const_node, node], [x], [y1, y2, y3, y4])
 
 
+@onnx_test()
+def split_test_uneven():
+    x = helper.make_tensor_value_info('x', TensorProto.FLOAT, [12, 15])
+    y1 = helper.make_tensor_value_info('y1', TensorProto.FLOAT, [3, 15])
+    y2 = helper.make_tensor_value_info('y2', TensorProto.FLOAT, [3, 15])
+    y3 = helper.make_tensor_value_info('y3', TensorProto.FLOAT, [3, 15])
+    y4 = helper.make_tensor_value_info('y4', TensorProto.FLOAT, [3, 15])
+    y5 = helper.make_tensor_value_info('y5', TensorProto.FLOAT, [0, 15])
+
+    node = onnx.helper.make_node(
+        'Split',
+        inputs=['x'],
+        outputs=['y1', 'y2', 'y3', 'y4', 'y5'],
+    )
+
+    return ([node], [x], [y1, y2, y3, y4, y5])
+
+
+@onnx_test()
+def split_test_uneven_num_outputs():
+    x = helper.make_tensor_value_info('x', TensorProto.FLOAT, [11, 15])
+    y1 = helper.make_tensor_value_info('y1', TensorProto.FLOAT, [3, 15])
+    y2 = helper.make_tensor_value_info('y2', TensorProto.FLOAT, [3, 15])
+    y3 = helper.make_tensor_value_info('y3', TensorProto.FLOAT, [3, 15])
+    y4 = helper.make_tensor_value_info('y4', TensorProto.FLOAT, [2, 15])
+
+    node = onnx.helper.make_node(
+        'Split',
+        inputs=['x'],
+        outputs=['y1', 'y2', 'y3', 'y4'],
+        num_outputs=4,
+    )
+
+    return ([node], [x], [y1, y2, y3, y4])
+
+
 @onnx_test()
 def split_test_no_attribute_invalid_split():
     x = helper.make_tensor_value_info('x', TensorProto.FLOAT, [300, 15])
@@ -8101,6 +8175,24 @@ def split_test_no_attribute_invalid_input_split():
     return ([node], [x], [y1, y2, y3])
 
 
+@onnx_test()
+def split_test_invalid_num_outputs():
+    x = helper.make_tensor_value_info('x', TensorProto.FLOAT, [11, 15])
+    y1 = helper.make_tensor_value_info('y1', TensorProto.FLOAT, [3, 15])
+    y2 = helper.make_tensor_value_info('y2', TensorProto.FLOAT, [3, 15])
+    y3 = helper.make_tensor_value_info('y3', TensorProto.FLOAT, [3, 15])
+    y4 = helper.make_tensor_value_info('y4', TensorProto.FLOAT, [2, 15])
+
+    node = onnx.helper.make_node(
+        'Split',
+        inputs=['x'],
+        outputs=['y1', 'y2', 'y3', 'y4'],
+        num_outputs=5,
+    )
+
+    return ([node], [x], [y1, y2, y3, y4])
+
+
 @onnx_test()
 def sqrt_test():
     x = helper.make_tensor_value_info('x', TensorProto.FLOAT, [10, 15])

test/onnx/onnx_test.cpp

@@ -4679,32 +4679,140 @@ TEST_CASE(multinomial_test)
 {
     migraphx::program p;
     auto* mm           = p.get_main_module();
-    size_t sample_size = 10;
-    float seed         = 0.0f;
+    size_t sample_size = 13;
+    size_t batch_size  = 3;
+    size_t categories  = 10;
+    float seed         = 0;
 
-    auto input = mm->add_parameter("input", migraphx::shape{migraphx::shape::float_type, {1, 10}});
-    auto maxes = mm->add_instruction(migraphx::make_op("reduce_max", {{"axes", {1}}}), input);
-    auto mb_maxes =
-        mm->add_instruction(migraphx::make_op("multibroadcast", {{"out_lens", {1, 10}}}), maxes);
+    auto input = mm->add_parameter(
+        "input", migraphx::shape{migraphx::shape::float_type, {batch_size, categories}});
+    auto maxes    = mm->add_instruction(migraphx::make_op("reduce_max", {{"axes", {1}}}), input);
+    auto mb_maxes = mm->add_instruction(
+        migraphx::make_op("multibroadcast", {{"out_lens", {batch_size, 10}}}), maxes);
     auto cdf = mm->add_instruction(migraphx::make_op("sub"), input, mb_maxes);
     cdf      = mm->add_instruction(migraphx::make_op("exp"), cdf);
     cdf      = mm->add_instruction(
         migraphx::make_op("prefix_scan_sum", {{"axis", 1}, {"exclusive", false}}), cdf);
 
-    std::mt19937 gen(seed);
-    std::uniform_real_distribution<> dis(0.0, 1.0);
-    std::vector<float> rand_samples(sample_size);
-    std::generate(rand_samples.begin(), rand_samples.end(), [&]() { return dis(gen); });
-    migraphx::shape rs{migraphx::shape::float_type, {1, sample_size}};
-    auto rs_lit = mm->add_literal(migraphx::literal{rs, rand_samples});
-
-    mm->add_instruction(migraphx::make_op("multinomial"), cdf, rs_lit);
+    migraphx::shape s{migraphx::shape::float_type, {1}};
+    std::vector<float> seed_data = {seed};
+    auto seed_input              = mm->add_literal(migraphx::literal(s, seed_data));
+    auto rand_dummy =
+        mm->add_literal(migraphx::literal{migraphx::shape::float_type, {batch_size * sample_size}});
 
+    auto randoms = mm->add_instruction(migraphx::make_op("random_uniform"), seed_input, rand_dummy);
+    mm->add_instruction(migraphx::make_op("multinomial"), cdf, randoms);
     auto prog = optimize_onnx("multinomial_test.onnx");
 
     EXPECT(p == prog);
 }
 
+TEST_CASE(multinomial_dyn_test)
+{
+    // compile-time random seed
+    migraphx::program p;
+    auto* mm           = p.get_main_module();
+    size_t sample_size = 100000;
+    size_t categories  = 5;
+    float seed         = 1.3f;
+
+    auto input = mm->add_parameter(
+        "input",
+        migraphx::shape{migraphx::shape::float_type, {{1, categories}, {categories, categories}}});
+
+    auto maxes = mm->add_instruction(migraphx::make_op("reduce_max", {{"axes", {1}}}), input);
+
+    auto cdf = add_common_op(*mm, migraphx::make_op("sub"), {input, maxes});
+    cdf      = mm->add_instruction(migraphx::make_op("exp"), cdf);
+    cdf      = mm->add_instruction(
+        migraphx::make_op("prefix_scan_sum", {{"axis", 1}, {"exclusive", false}}), cdf);
+
+    migraphx::shape s{migraphx::shape::float_type, {1}};
+    std::vector<float> seed_data = {seed};
+    auto seed_input              = mm->add_literal(migraphx::literal(s, seed_data));
+
+    // dynamic input only:  must calculate alloc_shape as (batch_size, sample_size)
+    //                read the runtime input dimensions
+    auto dim_of = mm->add_instruction(migraphx::make_op("dimensions_of", {{"end", 2}}), input);
+    // make an argument of (1, 0)
+    migraphx::shape lit_shape(migraphx::shape::int64_type, {2});
+    std::vector<int64_t> data1{1, 0};
+    auto l1        = mm->add_literal(lit_shape, data1);
+    auto batch_arg = mm->add_instruction(migraphx::make_op("mul"), dim_of, l1);
+    std::vector<int64_t> data2(2, 0);
+    // make an argument of (0, sample_size)
+    data2[1]         = sample_size;
+    auto l2          = mm->add_literal(lit_shape, data2);
+    auto alloc_shape = mm->add_instruction(migraphx::make_op("add"), batch_arg, l2);
+    migraphx::shape compile_shape =
+        migraphx::shape(migraphx::shape::float_type,
+                        {input->get_shape().dyn_dims().front(), {sample_size, sample_size}});
+
+    auto alloc = mm->add_instruction(
+        migraphx::make_op("allocate", {{"shape", to_value(compile_shape)}}), alloc_shape);
+
+    auto randoms = mm->add_instruction(migraphx::make_op("random_uniform"), seed_input, alloc);
+    auto ret     = mm->add_instruction(
+        migraphx::make_op("multinomial", {{"dtype", migraphx::shape::float_type}}), cdf, randoms);
+    mm->add_return({ret});
+
+    migraphx::onnx_options options;
+    options.default_dyn_dim_value  = {1, categories};
+    options.print_program_on_error = true;
+    auto prog                      = migraphx::parse_onnx("multinomial_dyn_test.onnx", options);
+    EXPECT(p == prog);
+}
+
+TEST_CASE(multinomial_autoseed_dyn_test)
+{
+    // runtime random seed
+    migraphx::program p;
+    auto* mm           = p.get_main_module();
+    size_t sample_size = 12;
+    size_t categories  = 10;
+
+    auto input = mm->add_parameter(
+        "input", migraphx::shape{migraphx::shape::float_type, {{1, 10}, {10, 10}}});
+
+    auto maxes = mm->add_instruction(migraphx::make_op("reduce_max", {{"axes", {1}}}), input);
+
+    auto cdf = add_common_op(*mm, migraphx::make_op("sub"), {input, maxes});
+    cdf      = mm->add_instruction(migraphx::make_op("exp"), cdf);
+    cdf      = mm->add_instruction(
+        migraphx::make_op("prefix_scan_sum", {{"axis", 1}, {"exclusive", false}}), cdf);
+    auto seed_input = mm->add_instruction(migraphx::make_op("random_seed"));
+
+    // dynamic input only:  must calculate alloc_shape as (batch_size, sample_size)
+    //                read the runtime input dimensions
+    auto dim_of = mm->add_instruction(migraphx::make_op("dimensions_of", {{"end", 2}}), input);
+    // make an argument of (1, 0)
+    migraphx::shape lit_shape(migraphx::shape::int64_type, {2});
+    std::vector<int64_t> data1{1, 0};
+    auto l1        = mm->add_literal(lit_shape, data1);
+    auto batch_arg = mm->add_instruction(migraphx::make_op("mul"), dim_of, l1);
+    std::vector<int64_t> data2(2, 0);
+    // make an argument of (0, sample_size)
+    data2[1]         = sample_size;
+    auto l2          = mm->add_literal(lit_shape, data2);
+    auto alloc_shape = mm->add_instruction(migraphx::make_op("add"), batch_arg, l2);
+    migraphx::shape compile_shape =
+        migraphx::shape(migraphx::shape::float_type,
+                        {input->get_shape().dyn_dims().front(), {sample_size, sample_size}});
+
+    auto alloc = mm->add_instruction(
+        migraphx::make_op("allocate", {{"shape", to_value(compile_shape)}}), alloc_shape);
+
+    auto randoms = mm->add_instruction(migraphx::make_op("random_uniform"), seed_input, alloc);
+    auto ret     = mm->add_instruction(migraphx::make_op("multinomial"), cdf, randoms);
+    mm->add_return({ret});
+
+    migraphx::onnx_options options;
+    options.default_dyn_dim_value  = {1, categories};
+    options.print_program_on_error = true;
+    auto prog = migraphx::parse_onnx("multinomial_autoseed_dyn_test.onnx", options);
+    EXPECT(p == prog);
+}
+
 TEST_CASE(multinomial_dtype_error_test)
 {
     EXPECT(test::throws([&] { migraphx::parse_onnx("multinomial_dtype_error_test.onnx"); }));
@@ -4712,10 +4820,11 @@ TEST_CASE(multinomial_dtype_error_test)
 
 TEST_CASE(multinomial_generated_seed_test)
 {
+    // multinomial op. no longer generates its own randoms
     auto p1 = optimize_onnx("multinomial_generated_seed_test.onnx");
     auto p2 = optimize_onnx("multinomial_generated_seed_test.onnx");
 
-    EXPECT(p1 != p2);
+    EXPECT(p1 == p2);
 }
 
 TEST_CASE(multinomial_int64_test)
@@ -4723,27 +4832,27 @@ TEST_CASE(multinomial_int64_test)
     migraphx::program p;
     auto* mm                      = p.get_main_module();
     size_t sample_size            = 10;
-    float seed                    = 1.0f;
+    float seed                    = 1.0;
+    uint32_t batch_size           = 1;
     migraphx::shape::type_t dtype = migraphx::shape::type_t::int64_type;
 
     auto input = mm->add_parameter("input", migraphx::shape{migraphx::shape::float_type, {1, 10}});
     auto maxes = mm->add_instruction(migraphx::make_op("reduce_max", {{"axes", {1}}}), input);
-    auto mb_maxes =
-        mm->add_instruction(migraphx::make_op("multibroadcast", {{"out_lens", {1, 10}}}), maxes);
-    auto cdf = mm->add_instruction(migraphx::make_op("sub"), input, mb_maxes);
+
+    auto cdf = add_common_op(*mm, migraphx::make_op("sub"), {input, maxes});
     cdf      = mm->add_instruction(migraphx::make_op("exp"), cdf);
     cdf      = mm->add_instruction(
         migraphx::make_op("prefix_scan_sum", {{"axis", 1}, {"exclusive", false}}), cdf);
 
-    std::mt19937 gen(seed);
-    std::uniform_real_distribution<> dis(0.0, 1.0);
-    std::vector<float> rand_samples(sample_size);
-    std::generate(rand_samples.begin(), rand_samples.end(), [&]() { return dis(gen); });
-    migraphx::shape rs{migraphx::shape::float_type, {1, sample_size}};
-    auto rs_lit = mm->add_literal(migraphx::literal{rs, rand_samples});
-
-    mm->add_instruction(migraphx::make_op("multinomial", {{"dtype", dtype}}), cdf, rs_lit);
+    migraphx::shape s{migraphx::shape::float_type, {1}};
+    std::vector<float> data = {seed};
+    auto seed_input         = mm->add_literal(migraphx::literal(s, data));
 
+    // static size
+    auto rand_dummy =
+        mm->add_literal(migraphx::literal{migraphx::shape::float_type, {batch_size * sample_size}});
+    auto randoms = mm->add_instruction(migraphx::make_op("random_uniform"), seed_input, rand_dummy);
+    mm->add_instruction(migraphx::make_op("multinomial", {{"dtype", dtype}}), cdf, randoms);
     auto prog = optimize_onnx("multinomial_int64_test.onnx");
 
     EXPECT(p == prog);
@@ -7671,6 +7780,46 @@ TEST_CASE(split_test_default)
     EXPECT(p == prog);
 }
 
+TEST_CASE(split_test_uneven)
+{
+    migraphx::program p;
+    auto* mm   = p.get_main_module();
+    auto input = mm->add_parameter("x", migraphx::shape{migraphx::shape::float_type, {12, 15}});
+    auto r1    = mm->add_instruction(
+        migraphx::make_op("slice", {{"axes", {0}}, {"starts", {0}}, {"ends", {3}}}), input);
+    auto r2 = mm->add_instruction(
+        migraphx::make_op("slice", {{"axes", {0}}, {"starts", {3}}, {"ends", {6}}}), input);
+    auto r3 = mm->add_instruction(
+        migraphx::make_op("slice", {{"axes", {0}}, {"starts", {6}}, {"ends", {9}}}), input);
+    auto r4 = mm->add_instruction(
+        migraphx::make_op("slice", {{"axes", {0}}, {"starts", {9}}, {"ends", {12}}}), input);
+    auto r5 = mm->add_instruction(
+        migraphx::make_op("slice", {{"axes", {0}}, {"starts", {12}}, {"ends", {12}}}), input);
+    mm->add_return({r1, r2, r3, r4, r5});
+
+    auto prog = migraphx::parse_onnx("split_test_uneven.onnx");
+    EXPECT(p == prog);
+}
+
+TEST_CASE(split_test_uneven_num_outputs)
+{
+    migraphx::program p;
+    auto* mm   = p.get_main_module();
+    auto input = mm->add_parameter("x", migraphx::shape{migraphx::shape::float_type, {11, 15}});
+    auto r1    = mm->add_instruction(
+        migraphx::make_op("slice", {{"axes", {0}}, {"starts", {0}}, {"ends", {3}}}), input);
+    auto r2 = mm->add_instruction(
+        migraphx::make_op("slice", {{"axes", {0}}, {"starts", {3}}, {"ends", {6}}}), input);
+    auto r3 = mm->add_instruction(
+        migraphx::make_op("slice", {{"axes", {0}}, {"starts", {6}}, {"ends", {9}}}), input);
+    auto r4 = mm->add_instruction(
+        migraphx::make_op("slice", {{"axes", {0}}, {"starts", {9}}, {"ends", {11}}}), input);
+    mm->add_return({r1, r2, r3, r4});
+
+    auto prog = migraphx::parse_onnx("split_test_uneven_num_outputs.onnx");
+    EXPECT(p == prog);
+}
+
 TEST_CASE(split_test_no_attribute_invalid_split)
 {
     EXPECT(
@@ -7688,6 +7837,11 @@ TEST_CASE(split_test_no_attribute_invalid_input_split)
         [&] { migraphx::parse_onnx("split_test_no_attribute_invalid_input_split.onnx"); }));
 }
 
+TEST_CASE(split_test_invalid_num_outputs)
+{
+    EXPECT(test::throws([&] { migraphx::parse_onnx("split_test_invalid_num_outputs.onnx"); }));
+}
+
 TEST_CASE(sqrt_test)
 {
     migraphx::program p;

test/onnx/split_test_invalid_num_outputs.onnx

+	split_test_invalid_num_outputs:

+.
+
xy1y2y3y4"Split*
+num_outputs
split_test_invalid_num_outputsZ
+
x
+

+
+b
+y1
+

+
+b
+y2
+

+
+b
+y3
+

+
+b
+y4
+

+
+B
\ No newline at end of file

test/onnx/split_test_uneven_num_outputs.onnx

+	split_test_uneven_num_outputs:

+.
+
xy1y2y3y4"Split*
+num_outputs
split_test_uneven_num_outputsZ
+
x
+

+
+b
+y1
+

+
+b
+y2
+

+
+b
+y3
+

+
+b
+y4
+

+
+B
\ No newline at end of file