Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
5f48dfc2
Commit
5f48dfc2
authored
Dec 21, 2021
by
Igor Ostrovsky
Browse files
Add BLiMP
parent
df5d7cf0
Changes
138
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
20 additions
and
0 deletions
+20
-0
tests/tests/testdata/blimp_distractor_agreement_relative_clause-v0-loglikelihood
...imp_distractor_agreement_relative_clause-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_distractor_agreement_relative_clause-v0-res.json
...ta/blimp_distractor_agreement_relative_clause-v0-res.json
+1
-0
tests/tests/testdata/blimp_drop_argument-v0-loglikelihood
tests/tests/testdata/blimp_drop_argument-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_drop_argument-v0-res.json
tests/tests/testdata/blimp_drop_argument-v0-res.json
+1
-0
tests/tests/testdata/blimp_ellipsis_n_bar_1-v0-loglikelihood
tests/tests/testdata/blimp_ellipsis_n_bar_1-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_ellipsis_n_bar_1-v0-res.json
tests/tests/testdata/blimp_ellipsis_n_bar_1-v0-res.json
+1
-0
tests/tests/testdata/blimp_ellipsis_n_bar_2-v0-loglikelihood
tests/tests/testdata/blimp_ellipsis_n_bar_2-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_ellipsis_n_bar_2-v0-res.json
tests/tests/testdata/blimp_ellipsis_n_bar_2-v0-res.json
+1
-0
tests/tests/testdata/blimp_existential_there_object_raising-v0-loglikelihood
...a/blimp_existential_there_object_raising-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_existential_there_object_raising-v0-res.json
...stdata/blimp_existential_there_object_raising-v0-res.json
+1
-0
tests/tests/testdata/blimp_existential_there_quantifiers_1-v0-loglikelihood
...ta/blimp_existential_there_quantifiers_1-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_existential_there_quantifiers_1-v0-res.json
...estdata/blimp_existential_there_quantifiers_1-v0-res.json
+1
-0
tests/tests/testdata/blimp_existential_there_quantifiers_2-v0-loglikelihood
...ta/blimp_existential_there_quantifiers_2-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_existential_there_quantifiers_2-v0-res.json
...estdata/blimp_existential_there_quantifiers_2-v0-res.json
+1
-0
tests/tests/testdata/blimp_existential_there_subject_raising-v0-loglikelihood
.../blimp_existential_there_subject_raising-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_existential_there_subject_raising-v0-res.json
...tdata/blimp_existential_there_subject_raising-v0-res.json
+1
-0
tests/tests/testdata/blimp_expletive_it_object_raising-v0-loglikelihood
...stdata/blimp_expletive_it_object_raising-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_expletive_it_object_raising-v0-res.json
...ts/testdata/blimp_expletive_it_object_raising-v0-res.json
+1
-0
tests/tests/testdata/blimp_inchoative-v0-loglikelihood
tests/tests/testdata/blimp_inchoative-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_inchoative-v0-res.json
tests/tests/testdata/blimp_inchoative-v0-res.json
+1
-0
No files found.
tests/tests/testdata/blimp_distractor_agreement_relative_clause-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
bf78e2b53c0f3531303c668c96bd3897a0a35e960da37439e63724ecba4e371a
\ No newline at end of file
tests/tests/testdata/blimp_distractor_agreement_relative_clause-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_distractor_agreement_relative_clause"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_distractor_agreement_relative_clause"
:
0
}}
\ No newline at end of file
tests/tests/testdata/blimp_drop_argument-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
616109e63f162dcd31a632943e7ef0c9e0431afeb179e83e9b04b39007b16f5b
\ No newline at end of file
tests/tests/testdata/blimp_drop_argument-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_drop_argument"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_drop_argument"
:
0
}}
\ No newline at end of file
tests/tests/testdata/blimp_ellipsis_n_bar_1-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
d14e4b7fcdd68991eb39b9cf3ade4b37dee9ddd39b688f861d81a327e47a969f
\ No newline at end of file
tests/tests/testdata/blimp_ellipsis_n_bar_1-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_ellipsis_n_bar_1"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_ellipsis_n_bar_1"
:
0
}}
\ No newline at end of file
tests/tests/testdata/blimp_ellipsis_n_bar_2-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
0523771a217759f0b22b89807694ee7f6381ce98a584b1fd070ba96194a3273b
\ No newline at end of file
tests/tests/testdata/blimp_ellipsis_n_bar_2-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_ellipsis_n_bar_2"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_ellipsis_n_bar_2"
:
0
}}
\ No newline at end of file
tests/tests/testdata/blimp_existential_there_object_raising-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
63567712076256f373131971676c1c6d711efef73cd0e4de3cc639bc631a2413
\ No newline at end of file
tests/tests/testdata/blimp_existential_there_object_raising-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_existential_there_object_raising"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_existential_there_object_raising"
:
0
}}
\ No newline at end of file
tests/tests/testdata/blimp_existential_there_quantifiers_1-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
d77594382e6d9af31a8b8ef00ba1ef6c29d6be6d0ddb7a9c27ef25ace654e05a
\ No newline at end of file
tests/tests/testdata/blimp_existential_there_quantifiers_1-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_existential_there_quantifiers_1"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_existential_there_quantifiers_1"
:
0
}}
\ No newline at end of file
tests/tests/testdata/blimp_existential_there_quantifiers_2-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
6e6add7baff4217f383425bef58288202018e041b24084edcaa5df8af08f820c
\ No newline at end of file
tests/tests/testdata/blimp_existential_there_quantifiers_2-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_existential_there_quantifiers_2"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_existential_there_quantifiers_2"
:
0
}}
\ No newline at end of file
tests/tests/testdata/blimp_existential_there_subject_raising-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
9b324b28ae3e1b5d49ecf4b7b2a16c7bbc8ff38d000cf216fab75df633da2084
\ No newline at end of file
tests/tests/testdata/blimp_existential_there_subject_raising-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_existential_there_subject_raising"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_existential_there_subject_raising"
:
0
}}
\ No newline at end of file
tests/tests/testdata/blimp_expletive_it_object_raising-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
ceede5b38248a62125a74a8332602b8eac5ef40864f071ad8d86e7971e07219d
\ No newline at end of file
tests/tests/testdata/blimp_expletive_it_object_raising-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_expletive_it_object_raising"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_expletive_it_object_raising"
:
0
}}
\ No newline at end of file
tests/tests/testdata/blimp_inchoative-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
3ff73629fb4473986a0e8ae2fcb7c40e88292189ab0d8755d20836c5aa5a2f99
\ No newline at end of file
tests/tests/testdata/blimp_inchoative-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_inchoative"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_inchoative"
:
0
}}
\ No newline at end of file
Prev
1
2
3
4
5
6
7
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment