Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
3fe5c8e8
"git@developer.sourcefind.cn:chenpangpang/transformers.git" did not exist on "ae5a32bb0d1c373fff9ee98c58a28c3396558d65"
Commit
3fe5c8e8
authored
Sep 19, 2019
by
VictorSanh
Browse files
update bert-base-uncased rslts
parent
354944e6
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
9 additions
and
9 deletions
+9
-9
examples/README.md
examples/README.md
+9
-9
No files found.
examples/README.md
View file @
3fe5c8e8
...
...
@@ -97,20 +97,20 @@ Fine-tuning the library models for sequence classification on the GLUE benchmark
Evaluation
](
https://gluebenchmark.com/
)
. This script can fine-tune the following models: BERT, XLM, XLNet and RoBERTa.
GLUE is made up of a total of 9 different tasks. We get the following results on the dev set of the benchmark with an
uncased BERT base model (the checkpoint
`bert-base-uncased`
). All experiments ran on 8
V100 GPUs with a total train
uncased BERT base model (the checkpoint
`bert-base-uncased`
). All experiments ran on 8 V100 GPUs with a total train
batch size of 24. Some of these tasks have a small dataset and training can lead to high variance in the results
between different runs. We report the median on 5 runs (with different seeds) for each of the metrics.
| Task | Metric | Result |
|-------|------------------------------|-------------|
| CoLA | Matthew's corr |
55.75
|
| SST-2 | Accuracy | 9
2.09
|
| MRPC | F1/Accuracy | 90.
48
/86.27 |
| STS-B | Person/Spearman corr. |
89.03/88.6
4 |
| QQP | Accuracy/F1 | 90.9
2
/87.
72
|
| MNLI | Matched acc./Mismatched acc. | 83.7
4
/84.
06
|
| QNLI | Accuracy |
91.07
|
| RTE | Accuracy |
68.59
|
| CoLA | Matthew's corr |
48.87
|
| SST-2 | Accuracy | 9
1.74
|
| MRPC | F1/Accuracy | 90.
70
/86.27 |
| STS-B | Person/Spearman corr. |
91.39/91.0
4 |
| QQP | Accuracy/F1 | 90.
7
9/87.
66
|
| MNLI | Matched acc./Mismatched acc. | 83.7
0
/84.
83
|
| QNLI | Accuracy |
89.31
|
| RTE | Accuracy |
71.43
|
| WNLI | Accuracy | 43.66 |
Some of these results are significantly different from the ones reported on the test set
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment