GitLab

**Description**
Add rccl bandwidth microbenchmark for rocm.

**Major Revision**
- Register rccl-bw benchmark.

**Description**
Add the source code of rocm kernel launch overhead benchmark. 

**Major Revision**
- Revise cmake build logic to support both cuda and rocm

**Description**
Support rocm cmake build. 

**Major Revision**
- Add  some envs in rocm_common.cmake to support rocm cmake build.

**Description**
Add NCCL performance microbenchmark.

**Major Revision**
- Add microbenchmark, example, test, config for NCCL

**Description**
Add RDMA Loopback performance microbenchmark.

**Major Revision**
- Add microbenchmark, example, test, config for RDMA Loopback

**Description**
Add disk performance microbenchmark.

**Major Revision**
- Add microbenchmark, example, test, config for disk performance.

**Minor Revision**
- Fix bugs in executor unit test related to default enabled tests.

Add microbenchmark, example, test, config for cuda memory performance and Add cuda-samples(tag with cuda version) as git submodule and update related makefile

* fix bug for nvidia v100
* hard code the supported dict for different arch.

Benchmarks: Add Feature - Add DistributedImpl and DistributedBackend arguments for micro benchmark. (#100)

rename bin name and result metric of cublas and cudnn microbenchmark

* Change default shape of sharding-matmul.

* add cuda flops performance benchmark.

* add python related cudnn microbenchmark

* add error return code for cublas micro benchmark

* Benchmarks: Add benchmark - add source code of cudnn function micro benchmark

* add benchmark for cublas test

* format

* revise error handling and test

* add interface to read json file, revise json file path and include .json in packaging

* add random_seed in arguments

* revise preprocess of cublas benchmark

* fix lint error and note error in source code

* update according comments

* revise input arguments from json file to custom str and convert json file to built-in dict list

* restore package config

* fit lint issue

* update platform and comments

* rename files to match source code dir and fix comments error
Co-authored-by: root <root@sb-validation-000001.51z1chmys5fuzfqyo4niepozre.bx.internal.cloudapp.net>

* Superbenchmark: Add benchmarks - add cublas function micro benchmark

* format

* add python benchmark for cublas functions, example and test file

* detele python related and rename some files

* revise cmd_helper and move json package to cmake

* resolve conflict

* revise error handing to try-catch and update some code style

* revise cmd_helper.h, cublas_helper.h, cublas_helper.cpp

* revise structure of the cublas function

* add some comments and move cuda_init and cuda_free

* add comments for class member

* add ramdom seed, revise input from file to json string, simplify cmake

* delete json file in source code of cublas

* update according comments

* limit batchcount=1 in initialization of cublas function which do not use batch count

* revise and fix some errors of annotations

* update according comments and revise construction of CublasFunction
Co-authored-by: root <root@sb-validation-000001.51z1chmys5fuzfqyo4niepozre.bx.internal.cloudapp.net>

* Unify SB_MICRO_PATH and SB_MICRO_LIB

* fix bug of lib path

* add kernel launch overhead benchmark.

* add cuda kernel launch overhead benchmark - source part.
* can customize the nvcc_archs_support.
* set SB_MICRO_PATH for azure pipeline tests.

* Revise MicroBenchmark class to be more flexible.
* use command index not the command as the parameter.
* changes according to discussion.

* Benchmarks: Add Benchmark - add computation and communication overlap micro benchmark

* Benchmarks: Add benchmark - fix some format issues and typo

* Benchmarks: Add Benchmark - update according comments and add test

* revise tests

* skip multi gpu test due to no multi gpu
Co-authored-by: v-yujiang <v-yujiang@microsoft.com>

Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>

Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>

Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>

Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>

Benchmarks: Add Benchmark - Add op-sharding microbenchmark, including matmul and sharding_matmul. (#36)

* add microbenchmark - sharding matmul.
* address comments.
Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>

* revise result process interface

* add more comments
Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>

* move benchmarks registration from registry.py to __init__.py

* revise __init__.
Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>

* benchmarks init.
Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>