2022-04-28-release-0-5.md 1.99 KB
Newer Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
---
slug: release-sb-v0.5
title: Releasing SuperBench v0.5
author: Peng Cheng
author_title: SuperBench Team
author_url: https://github.com/cp5555
author_image_url: https://github.com/cp5555.png
tags: [superbench, announcement, release]
---

We are very happy to announce that **SuperBench 0.5.0 version** is officially released today!

You can install and try superbench by following [Getting Started Tutorial](https://microsoft.github.io/superbenchmark/docs/getting-started/installation).

## SuperBench 0.5.0 Release Notes

### Micro-benchmark Improvements

- Support NIC only NCCL bandwidth benchmark on single node in NCCL/RCCL bandwidth test.
- Support bi-directional bandwidth benchmark in GPU copy bandwidth test.
- Support data checking in GPU copy bandwidth test.
- Update rccl-tests submodule to fix divide by zero error.
- Add GPU-Burn micro-benchmark.

### Model-benchmark Improvements

- Sync results on root rank for e2e model benchmarks in distributed mode.
- Support customized `env` in local and torch.distributed mode.
- Add support for pytorch>=1.9.0.
- Keep BatchNorm as fp32 for pytorch cnn models cast to fp16.
- Remove FP16 samples type converting time.
- Support FAMBench.

### Inference Benchmark Improvements

- Revise the default setting for inference benchmark.
- Add percentile metrics for inference benchmarks.
- Support T4 and A10 in GEMM benchmark.
- Add configuration with inference benchmark.

### Other Improvements

- Add command to support listing all optional parameters for benchmarks.
- Unify benchmark naming convention and support multiple tests with same benchmark and different parameters/options in one configuration file.
- Support timeout to detect the benchmark failure and stop the process automatically.
- Add rocm5.0 dockerfile.
- Improve output interface.

### Data Diagnosis and Analysis

- Support multi-benchmark check.
- Support result summary in md, html and excel formats.
- Support data diagnosis in md and html formats.
- Support result output for all nodes in data diagnosis.