README.md 1.25 KB
Newer Older
1
2
## Overview

digger yu's avatar
digger yu committed
3
This directory includes two parts: Using the Booster API finetune Huggingface Bert and AlBert models and benchmarking Bert and AlBert models with different Booster Plugin.
4
5
6
7
8
9

## Finetune
```
bash test_ci.sh
```

10
11
12
13
14
15
16
17
### Bert-Finetune Results

| Plugin         | Accuracy | F1-score | GPU number |
| -------------- | -------- | -------- | -------- |
| torch_ddp      | 84.4%    | 88.6%    |    2     |
| torch_ddp_fp16 | 84.7%    | 88.8%    |    2     |
| gemini         | 84.0%    | 88.4%    |    2     |
| hybrid_parallel | 84.5%    | 88.6%    |    4     |
18
19


20
21
22
23
24
25
26
## Benchmark
```
bash benchmark.sh
```

Now include these metrics in benchmark: CUDA mem occupy, throughput and the number of model parameters. If you have custom metrics, you can add them to benchmark_util.

27
### Results
28

29
#### Bert
30
31
32
33
34
35
36
37

|       | max cuda mem | throughput(sample/s) | params |
| :-----| -----------: | :--------: | :----: |
| ddp | 21.44 GB | 3.0 | 82M |
| ddp_fp16 | 16.26 GB | 11.3 | 82M |
| gemini | 11.0 GB | 12.9 | 82M |
| low_level_zero | 11.29 G | 14.7 | 82M |

38
#### AlBert
39
40
41
42
43
|       | max cuda mem | throughput(sample/s) | params |
| :-----| -----------: | :--------: | :----: |
| ddp | OOM |  | |
| ddp_fp16 | OOM |  | |
| gemini | 69.39 G | 1.3 | 208M |
44
| low_level_zero | 56.89 G | 1.4 | 208M |