GitLab

__Description__

Upgrade version and release note.

__Major Revision__

- Upgrade package versions
- Add release note for v0.4.0

**Description**
Update inference and network benchmarks in configs.

**Description**
Launch mpi on the sorted first host in the hostfile.

**Description**
Fix bugs in data diagnosis.

**Major Revision**
- fix package import issue of file_handler
- deal with monitor metrics
- fix typo in output_path

**Description**
Fix bug of detecting if gpu_index is none.

**Description**
Add 'monitor/' prefix to monitor metrics in result summary.

**Description**
Unify metric and add doc for cuBLAS and cuDNN functions.

**Description**
Fix fio build issue (Illegal instruction).
Refer to https://github.com/axboe/fio/issues/970

**Description**
Build openmpi with ucx support in rocm dockerfiles.

__Description__

Refine test cases for microbenchmark:
* Refine test fixture, add BenchmarkTestCase class.
* Refine test data.
* Resolve no numa issue for test_ib_loopback_util case.

__Description__

Fix issues for Ansible and benchmarks:
* Cleanup Ansible runner private data dir to avoid out of disk space issue when node number is large.
* Support both absolute and relative paths when fecth results.
* Use a deterministic image in Ansible test to avoid image update.
* Update logging format.
* Delete torch models and inputs after export.

**Description**
Add usage for data diagnosis.

**Description**
Update docs for monitor.

Add transformers for TensorRT inference.

**Description**
Add benchmark metrics for cpu-memory-bw-latency.

**Description**
Benchmarks: Fix Comment - Correct benchmark name in test_gpu_copy_bw_performance.py.

**Description**
Add mlc memory bandwidth and latency micro benchmark to Superbench.

**Major Revision**
- Add mlc benchmark with test and example files

**Description**
Minor doc change to highlight sb CLI version is independent of the sb container version.

**Description**
Add ONNXRuntime inference benchmark based on ORT python API.

**Major Revision**
- Add `ORTInferenceBenchmark` class to export pytorch model to onnx model and do inference
- Add tests and example for `ort-inference` benchmark
- Update the introduction docs.

**Description**
Add basic analysis features.

**Major Revision**
- Add statistics, correlations of the raw data
- Add numeric outlier detection(inter_quartile_range)
- Add boxplot for selected metric

**Description**
Integrate monitor into Superbench.

**Major Revision**
- Initialize, start and stop monitor in SB executor.
- Parse the monitor data in SB runner and merge into benchmark results.
- Specify ReduceType for monitor metrics, such as MAX, MIN and LAST.
- Add monitor configs into config file.

**Description**
Set the `reduce_op` type for metirc `return_code` as `None`.

**Description**
Add cli to integrate data diagnosis module.

**Description**
Unify metric names of benchmarks.

**Description**
Add data diagnosis module.

**Major Revision**
- Add DataDiagnosis class to support rule-based data diagnosis for result summary jsonl file of multi nodes
- Add RuleOp class to define rule operators

Fix issues for distributed runs:
* fix config for memory bandwidth benchmarks
* add throttling for high concurrency docker pull
* update rsync path and exclude directories
* handle exceptions when creating summary
* tune for logging

**Description**
Add return_code metric into result and revise unit tests.

**Description**
Add doc for data diagnosis, including input, output and baseline file schema.

Add config file for Azure NDm A100 v4 SKU.

**Description**
Add gpt-small into config files.

**Description**
If `ignore_invalid` is True, and 'required' arguments are not set when register the benchmark, the arguments should be provided by user in config and skip the arguments checking.

**Description**
Replace `-c` argument with `-N` for `numactl` since the old `-c`/`--cpubind` argument is deprecated.

**Description**
Upgrade FIO benchmark tool from 3.27 to 3.28.

**Description**
Update ib validtion mirobenchmark metrics.

Bumps [algoliasearch-helper](https://github.com/algolia/algoliasearch-helper-js) from 3.5.5 to 3.6.2.
- [Release notes](https://github.com/algolia/algoliasearch-helper-js/releases)
- [Changelog](https://github.com/algolia/algoliasearch-helper-js/blob/develop/CHANGELOG)
- [Commits](https://github.com/algolia/algoliasearch-helper-js/compare/3.5.5...3.6.2

)

---
updated-dependencies:
- dependency-name: algoliasearch-helper
  dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

**Description**

Update gpu-copy benchmark metrics.

**Description**
Fix typo in description of kernel_launch_overhead.py

**Description**
Add the initial version of Monitor.

**Major Revision**
- Add `Monitor` class to launch background process for monitoring.
- Add `MonitorRecord` class to save the data one time capturing.

**Description**
Rename `nvidia_helper` utility as `device_manager` module and support more functions:
```
device_manager.get_device_count()
device_manager.get_device_utilization(idx)
device_manager.get_device_temperature(idx)
device_manager.get_device_power_limit(idx)
device_manager.get_device_memory(idx)
device_manager.get_device_row_remapped_info(idx)
device_manager.get_device_ecc_error(idx)
```

__Description__

Add TensorRT inference benchmark for torchvision models.

__Major Revision__
- Measure TensorRT inference performance.