Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
84aa15c6
"mmdet3d/models/vscode:/vscode.git/clone" did not exist on "681024412529ff8a6485c71754ce8212469ea3dd"
Commit
84aa15c6
authored
Oct 27, 2021
by
Leo Gao
Browse files
Fix testdata for truthfulqa
parent
b67aec37
Changes
4
Hide whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
4 additions
and
4 deletions
+4
-4
tests/testdata/truthfulqa_gen-v0-greedy_until
tests/testdata/truthfulqa_gen-v0-greedy_until
+1
-1
tests/testdata/truthfulqa_gen-v0-res.json
tests/testdata/truthfulqa_gen-v0-res.json
+1
-1
tests/testdata/truthfulqa_mc-v0-loglikelihood
tests/testdata/truthfulqa_mc-v0-loglikelihood
+1
-1
tests/testdata/truthfulqa_mc-v0-res.json
tests/testdata/truthfulqa_mc-v0-res.json
+1
-1
No files found.
tests/testdata/truthfulqa_gen-v0-greedy_until
View file @
84aa15c6
1a280973bbac2b7ac29dd64dddac474fb4749585f7de893483b4034814466c67
0d7c56e1aa71ffd8f94bde28f6e8dfdd35f7aaadffa0620bd2a27704253d6c14
\ No newline at end of file
\ No newline at end of file
tests/testdata/truthfulqa_gen-v0-res.json
View file @
84aa15c6
{
"results"
:
{
"truthfulqa_gen"
:
{
"bleu_acc"
:
0.0
,
"bleu_acc_stderr"
:
0.0
,
"bleu_diff"
:
0.0
,
"bleu_diff_stderr"
:
0.0
,
"bleu_max"
:
0.0
,
"bleu_max_stderr"
:
0.0
,
"bleurt_acc"
:
0.835985312117503
,
"bleurt_acc_stderr"
:
0.012962704327492454
,
"bleurt_diff"
:
0.14077322143090107
,
"bleurt_diff_stderr"
:
0.005459888909582694
,
"bleurt_max"
:
-1.4399358725752065
,
"bleurt_max_stderr"
:
0.0022126992369197133
,
"rouge1_acc"
:
0.0
,
"rouge1_acc_stderr"
:
0.0
,
"rouge1_diff"
:
0.0
,
"rouge1_diff_stderr"
:
0.0
,
"rouge1_max"
:
0.0
,
"rouge1_max_stderr"
:
0.0
,
"rouge2_acc"
:
0.0
,
"rouge2_acc_stderr"
:
0.0
,
"rouge2_diff"
:
0.0
,
"rouge2_diff_stderr"
:
0.0
,
"rouge2_max"
:
0.0
,
"rouge2_max_stderr"
:
0.0
,
"rougeL_acc"
:
0.0
,
"rougeL_acc_stderr"
:
0.0
,
"rougeL_diff"
:
0.0
,
"rougeL_diff_stderr"
:
0.0
,
"rougeL_max"
:
0.0
,
"rougeL_max_stderr"
:
0.0
}},
"versions"
:
{
"truthfulqa_gen"
:
0
}}
{
"results"
:
{
"truthfulqa_gen"
:
{
"bleu_acc"
:
0.0
,
"bleu_acc_stderr"
:
0.0
,
"bleu_diff"
:
0.0
,
"bleu_diff_stderr"
:
0.0
,
"bleu_max"
:
0.0
,
"bleu_max_stderr"
:
0.0
,
"bleurt_acc"
:
0.8372093023255814
,
"bleurt_acc_stderr"
:
0.012923696051772253
,
"bleurt_diff"
:
0.13967358205134603
,
"bleurt_diff_stderr"
:
0.00532907098769571
,
"bleurt_max"
:
-1.4402793981454072
,
"bleurt_max_stderr"
:
0.0021884846359458963
,
"rouge1_acc"
:
0.0
,
"rouge1_acc_stderr"
:
0.0
,
"rouge1_diff"
:
0.0
,
"rouge1_diff_stderr"
:
0.0
,
"rouge1_max"
:
0.0
,
"rouge1_max_stderr"
:
0.0
,
"rouge2_acc"
:
0.0
,
"rouge2_acc_stderr"
:
0.0
,
"rouge2_diff"
:
0.0
,
"rouge2_diff_stderr"
:
0.0
,
"rouge2_max"
:
0.0
,
"rouge2_max_stderr"
:
0.0
,
"rougeL_acc"
:
0.0
,
"rougeL_acc_stderr"
:
0.0
,
"rougeL_diff"
:
0.0
,
"rougeL_diff_stderr"
:
0.0
,
"rougeL_max"
:
0.0
,
"rougeL_max_stderr"
:
0.0
}},
"versions"
:
{
"truthfulqa_gen"
:
0
}}
\ No newline at end of file
\ No newline at end of file
tests/testdata/truthfulqa_mc-v0-loglikelihood
View file @
84aa15c6
1e07020e9cf41d46ed65312eb39d2b8e6599673d4f0d6b67c0d0eba0efb493bb
226a6783976177dc9ceda5688623ff37023242eff30ddf270b886bf7b9b32228
\ No newline at end of file
\ No newline at end of file
tests/testdata/truthfulqa_mc-v0-res.json
View file @
84aa15c6
{
"results"
:
{
"truthfulqa_mc"
:
{
"mc1"
:
0.23255813953488372
,
"mc1_stderr"
:
0.01478915753108052
,
"mc2"
:
0.4462325560722362
,
"mc2_stderr"
:
0.004986523944692003
}},
"versions"
:
{
"truthfulqa_mc"
:
0
}}
{
"results"
:
{
"truthfulqa_mc"
:
{
"mc1"
:
0.2141982864137087
,
"mc1_stderr"
:
0.01436214815569045
,
"mc2"
:
0.465436996173817
,
"mc2_stderr"
:
0.0048422530880316405
}},
"versions"
:
{
"truthfulqa_mc"
:
0
}}
\ No newline at end of file
\ No newline at end of file
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment