Commits · acf365a8567a7935fdf38d7b21458d1dbeed0108 · tsoc / superbenchmark

06 Aug, 2021 2 commits
- Benchmarks: Add Feature - Set reduce type for current benchmarks' metrics. (#149) · acf365a8
  guoshzhao authored Aug 06, 2021
```
**Description**
Set reduce type for current benchmarks' metrics, including model benchmarks and ShardingMatmul.
```
  acf365a8
- Benchmarks: Code Revision - Calculate average value by using statistics module. (#148) · bc1a61b9
  guoshzhao authored Aug 06, 2021
```
**Description**
Replace `sum(results) / len(results)` with `statistics.mean(results)`
```
  bc1a61b9
05 Aug, 2021 1 commit

Benchmarks: Add Feature - Add reduce function support for output summary. (#147) · e41b1f62

guoshzhao authored Aug 05, 2021

**Description**
Add reduce function support for output summary.

**Major Revision**
- Add reducer class to maintain all reduce functions.
- Save reduce type of each metric into `BenchmarkResult`
- Fix UT.

e41b1f62

28 Jun, 2021 1 commit
- Benchmarks: Code Revision - Replace torch.optim.AdamW with transformers.AdamW. (#106) · 9c748527
  guoshzhao authored Jun 28, 2021
```
* replace torch.optim.AdamW with transformers.AdamW.
```
  9c748527
07 Jun, 2021 1 commit
- Benchmarks: Fix Bug - Fix OOM issue when run pytorch models sequentially. (#93) · 03b41be1
  guoshzhao authored Jun 07, 2021
```
* Clean up the cache.
```
  03b41be1
04 Jun, 2021 1 commit
- Benchmarks: Fix Bug - Fix return code overwrite issue (#94) · 2d9be807
  guoshzhao authored Jun 04, 2021
```
* fix return code reset issue
```
  2d9be807
19 May, 2021 1 commit
- expose interface of pin memory and modify cnn configuration (#75) · b7d0ee32
  Yuting Jiang authored May 19, 2021
  
  b7d0ee32
20 Apr, 2021 2 commits
- Benchmarks: Add Benchmark - Add LSTM model benchmarks. (#60) · 2a7ab691
  guoshzhao authored Apr 20, 2021
```
* Benchmarks: Add Benchmark - Add LSTM model benchmarks.
```
  2a7ab691
- Benchmarks: Add Benchmark - Add CNN model benchmarks. (#59) · 902ea211
  guoshzhao authored Apr 20, 2021
```
* Benchmarks: Add Benchmark - Add CNN model benchmarks.
```
  902ea211
16 Apr, 2021 2 commits
- Benchmarks: Code Revision - Fix some issue for BERT benchmark. (#58) · ce3ed24a
  guoshzhao authored Apr 16, 2021
```
Benchmarks: Code Revision - Fix some issue for BERT benchmark. (#58)
```
  ce3ed24a
- Benchmarks: Add Benchmark - Add GPT2 model benchmark. (#57) · af567cf6
  guoshzhao authored Apr 16, 2021
```
* Benchmarks: Add Benchmark - Add GPT2 model benchmark.
```
  af567cf6
12 Apr, 2021 2 commits
- unify arguments format by using whitespace. (#50) · 4664019a
  guoshzhao authored Apr 12, 2021
```
Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>
Co-authored-by: Yifan Xiong <yifan.xiong@microsoft.com>
```
  4664019a
- Skip tests and remove useless tests (#42) · 7c0534cc
  Yifan Xiong authored Apr 12, 2021
```
* skip unnecessary tests according to env var
* remove useless tests
```
  7c0534cc
08 Apr, 2021 1 commit

Benchmarks: Code Revision - Revise BenchmarkRegistry interfaces for... · 923ce277

guoshzhao authored Apr 08, 2021


Benchmarks: Code Revision - Revise BenchmarkRegistry interfaces for integration with executor. (#33)

* revise BenchmarkRegistry interfaces.
* address comments
Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>

923ce277

26 Mar, 2021 1 commit

Benchmarks: Add Benchmark - Add Pytorch BERT benchmarks, including bert-base... · 0972b223

guoshzhao authored Mar 26, 2021


Benchmarks: Add Benchmark - Add Pytorch BERT benchmarks, including bert-base and bert-large.   (#20)

* add pytorch bert benchmarks.

* revise code

* fix issue

* revise code.
Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>

0972b223

22 Mar, 2021 1 commit
- Benchmarks: Code Revision - Move benchmarks auto-registration from registry.py to __init__.py (#24) · 8d24d03d
  guoshzhao authored Mar 22, 2021
```
* move benchmarks registration from registry.py to __init__.py

* revise __init__.
Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>
```
  8d24d03d
18 Mar, 2021 2 commits
- Benchmarks: Add Feature - Add sample_count argument for ModelBenchmark. (#22) · c00dc670
  guoshzhao authored Mar 18, 2021
```
* add sample_count argument.

* handle more condidatins.

* address comments.
Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>
```
  c00dc670
- Benchmarks: Code Revision - Support benchmark re-registration, keep the latest one. (#23) · 31b6f085
  guoshzhao authored Mar 18, 2021
```
* support benchmark re-registration.

* address comments
Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>
```
  31b6f085
17 Mar, 2021 1 commit
- Benchmarks: Add Test - Add tests for PytorchBase module. (#18) · 5b9b5cc8
  guoshzhao authored Mar 17, 2021
```
* add pytorch base tests.

* add more test cases.
Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>
```
  5b9b5cc8
11 Mar, 2021 1 commit

Benchmarks: Add Feature - Add random dataset for Pytorch. (#17) · ebea2d50

guoshzhao authored Mar 12, 2021



* add random dataset.

* install pytorch-cpu for test docker.

* fix typo

* add more test cases.

* address comments.
Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>

ebea2d50

09 Mar, 2021 2 commits
- Benchmarks: Add Feature - Add flag to disable GPU. (#15) · 52848d2f
  guoshzhao authored Mar 10, 2021
```
* add flag to disable GPU.

* fix spelling

* fix unittest.

* address comments.
Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>
```
  52848d2f
- rename _cal_params_size as _cal_params_count. (#16) · 83a4e93f
  guoshzhao authored Mar 09, 2021
```
Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>
```
  83a4e93f
04 Mar, 2021 1 commit
- add more checks for model base (#12) · 9388f8f5
  guoshzhao authored Mar 04, 2021
```
Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>
```
  9388f8f5
24 Feb, 2021 1 commit
- Benchmarks: Initialization - Add base class, registry, and result (#1) · 4c87a3e4
  guoshzhao authored Feb 24, 2021
```
* benchmarks init.
Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>
```
  4c87a3e4