Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
0538d4c9
Commit
0538d4c9
authored
Oct 06, 2021
by
Leo Gao
Browse files
Add truthfulqa test data
parent
dd59b0ef
Changes
4
Hide whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
4 additions
and
0 deletions
+4
-0
tests/testdata/truthfulqa_gen-v0-greedy_until
tests/testdata/truthfulqa_gen-v0-greedy_until
+1
-0
tests/testdata/truthfulqa_gen-v0-res.json
tests/testdata/truthfulqa_gen-v0-res.json
+1
-0
tests/testdata/truthfulqa_mc-v0-loglikelihood
tests/testdata/truthfulqa_mc-v0-loglikelihood
+1
-0
tests/testdata/truthfulqa_mc-v0-res.json
tests/testdata/truthfulqa_mc-v0-res.json
+1
-0
No files found.
tests/testdata/truthfulqa_gen-v0-greedy_until
0 → 100644
View file @
0538d4c9
1a280973bbac2b7ac29dd64dddac474fb4749585f7de893483b4034814466c67
\ No newline at end of file
tests/testdata/truthfulqa_gen-v0-res.json
0 → 100644
View file @
0538d4c9
{
"results"
:
{
"truthfulqa_gen"
:
{
"bleu acc"
:
0.0
,
"bleu acc_stderr"
:
0.0
,
"bleu diff"
:
0.0
,
"bleu diff_stderr"
:
0.0
,
"bleu max"
:
0.0
,
"bleu max_stderr"
:
0.0
,
"bleurt acc"
:
0.835985312117503
,
"bleurt acc_stderr"
:
0.012962704327492454
,
"bleurt diff"
:
0.14077322143090107
,
"bleurt diff_stderr"
:
0.005459888909582694
,
"bleurt max"
:
-1.4399358725752065
,
"bleurt max_stderr"
:
0.0022126992369197133
,
"rouge1 acc"
:
0.0
,
"rouge1 acc_stderr"
:
0.0
,
"rouge1 diff"
:
0.0
,
"rouge1 diff_stderr"
:
0.0
,
"rouge1 max"
:
0.0
,
"rouge1 max_stderr"
:
0.0
,
"rouge2 acc"
:
0.0
,
"rouge2 acc_stderr"
:
0.0
,
"rouge2 diff"
:
0.0
,
"rouge2 diff_stderr"
:
0.0
,
"rouge2 max"
:
0.0
,
"rouge2 max_stderr"
:
0.0
,
"rougeL acc"
:
0.0
,
"rougeL acc_stderr"
:
0.0
,
"rougeL diff"
:
0.0
,
"rougeL diff_stderr"
:
0.0
,
"rougeL max"
:
0.0
,
"rougeL max_stderr"
:
0.0
}},
"versions"
:
{
"truthfulqa_gen"
:
0
}}
\ No newline at end of file
tests/testdata/truthfulqa_mc-v0-loglikelihood
0 → 100644
View file @
0538d4c9
1e07020e9cf41d46ed65312eb39d2b8e6599673d4f0d6b67c0d0eba0efb493bb
\ No newline at end of file
tests/testdata/truthfulqa_mc-v0-res.json
0 → 100644
View file @
0538d4c9
{
"results"
:
{
"truthfulqa_mc"
:
{
"mc1"
:
0.23255813953488372
,
"mc1_stderr"
:
0.01478915753108052
,
"mc2"
:
0.4462325560722362
,
"mc2_stderr"
:
0.004986523944692003
}},
"versions"
:
{
"truthfulqa_mc"
:
0
}}
\ No newline at end of file
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment