Unverified Commit 6ecded20 authored by Yuting Jiang's avatar Yuting Jiang Committed by GitHub
Browse files

Docs - Unify metric and add doc for cublas and cudnn functions (#271)

**Description**
Unify metric and add doc for cuBLAS and cuDNN functions.
parent 74625527
......@@ -60,11 +60,40 @@ Large scale matmul operation using `torch.matmul` with one GPU.
### `cublas-function`
TODO
#### Introduction
Measure the performance of most common Nvidia cuBLAS functions with parameters in models training including ResNet, VGG, DenseNet, LSTM, BERT, and GPT-2.
The supported functions for cuBLAS are as follows:
- cublasSgemm
- cublasSgemmStridedBatched
- cublasGemmStridedBatchedEx
- cublasGemmEx
- cublasCgemm3mStridedBatched
- cublasCgemm
#### Metrics
| Name | Unit | Description |
|----------------------------------------------------------|-----------|-------------------------------------------------------------------|
| cublas-function/name_${function_name}_${parameters}_time | time (us) | The mean time to execute the cublas function with the parameters. |
### `cudnn-function`
TODO
#### Introduction
Measure the performance of most common Nvidia cuDNN functions with parameters in models training including ResNet, VGG, DenseNet, LSTM, BERT, and GPT-2.
The supported functions for cuDNN are as follows:
- cudnnConvolutionBackwardFilter
- cudnnConvolutionBackwardData
- cudnnConvolutionForward
#### Metrics
| Name | Unit | Description |
|---------------------------------------------------------|-----------|------------------------------------------------------------------|
| cudnn-function/name_${function_name}_${parameters}_time | time (us) | The mean time to execute the cudnn function with the parameters. |
### `tensorrt-inference`
......
......@@ -291,8 +291,8 @@ def _process_raw_result(self, cmd_idx, raw_output):
raw_data = raw_data.split(',')
raw_data.pop()
raw_data = [float(item) for item in raw_data]
self._result.add_result(metric, statistics.mean(raw_data))
self._result.add_raw_data(metric, raw_data)
self._result.add_result(metric.lower() + '_time', statistics.mean(raw_data))
self._result.add_raw_data(metric.lower() + '_time', raw_data)
if 'Error' in line:
error = True
except BaseException as e:
......
......@@ -6,6 +6,7 @@
import os
import json
import yaml
import statistics
from superbench.common.utils import logger
from superbench.benchmarks import Platform, BenchmarkRegistry, ReturnCode
......@@ -424,8 +425,8 @@ def _process_raw_result(self, cmd_idx, raw_output):
raw_data = raw_data.split(',')
raw_data.pop()
raw_data = [float(item) for item in raw_data]
self._result.add_result(metric, sum(raw_data) / len(raw_data))
self._result.add_raw_data(metric, raw_data)
self._result.add_result(metric.lower() + '_time', statistics.mean(raw_data) * 1000)
self._result.add_raw_data(metric.lower() + '_time', raw_data)
if 'Error' in line:
error = True
except BaseException as e:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment