Unverified Commit 682ed06a authored by Yuting Jiang's avatar Yuting Jiang Committed by GitHub
Browse files

Docs - Add usage for data diagnosis (#266)

**Description**
Add usage for data diagnosis.
parent 2e10fb0d
...@@ -101,6 +101,51 @@ Execute GPT2 model benchmark in default configuration: ...@@ -101,6 +101,51 @@ Execute GPT2 model benchmark in default configuration:
sb exec --config-override superbench.enable="['gpt2_models']" sb exec --config-override superbench.enable="['gpt2_models']"
``` ```
### `sb result diagnosis`
Filter the defective machines automatically from benchmarking results according to rules defined in rule file.
```bash title="SB CLI"
sb result diagnosis [--baseline-file]
[--data-file]
[--rule-file]
[--output-dir]
[--output-file-format]
```
#### Required arguments
| Name | Description |
|------------------------|------------------------|
| `--baseline-file` `-b` | Path to baseline file. |
| `--data-file` `-d` | Path to raw data file. |
| `--rule-file` `-r` | Path to rule file. |
#### Optional arguments
| Name | Default | Description |
|------------------------|---------|-----------------------------------------------------------------------------|
| `--output-dir` | `None` | Path to output directory, outputs/{datetime} will be used if not specified. |
| `--output-file-format` | `excel` | Format of output file, excel or json. Default: excel. |
#### Global arguments
| Name | Default | Description |
|---------------|---------|--------------------|
| `--help` `-h` | N/A | Show help message. |
#### Examples
Run data diagnosis and output the results in excel format:
```bash title="SB CLI"
sb result diagnosis --data-file outputs/results-summary.jsonl --rule-file rule.yaml --baseline-file baseline.json --output-file-foramt excel
```
Run data diagnosis and output the results in jsonl format:
```bash title="SB CLI"
sb result diagnosis --data-file outputs/results-summary.jsonl --rule-file rule.yaml --baseline-file baseline.json --output-file-foramt json
```
### `sb run` ### `sb run`
Run the SuperBench benchmarks distributedly. Run the SuperBench benchmarks distributedly.
...@@ -172,3 +217,4 @@ Print version: ...@@ -172,3 +217,4 @@ Print version:
```bash title="SB CLI" ```bash title="SB CLI"
sb version sb version
``` ```
...@@ -8,6 +8,20 @@ id: data-diagnosis ...@@ -8,6 +8,20 @@ id: data-diagnosis
This tool is to filter the defective machines automatically from thousands of benchmarking results according to rules defined in **rule file**. This tool is to filter the defective machines automatically from thousands of benchmarking results according to rules defined in **rule file**.
## Usage
1. [Install SuperBench](../getting-started/installation) on the local machine.
2. Prepare the raw data, rule file, baseline file under current path or somewhere on the local machine.
3. After installing the Superbnech and the files are ready, you can start to filter the defective machines automatically using `sb result diagnosis` command. The detailed command can be found from [SuperBench CLI](../cli).
```
sb result diagnosis --data-file ./results-summary.jsonl --rule-file ./rule.yaml --baseline-file ./baseline.json --output-file-foramt excel --output-dir ${output-dir}
```
4. After the command finished, you can find the output result file named 'diagnosis_summary.xlsx' / 'diagnosis_summary.json' under ${output_dir}.
## Input ## Input
The input mainly includes 3 files: The input mainly includes 3 files:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment