Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
5f48dfc2
Commit
5f48dfc2
authored
Dec 21, 2021
by
Igor Ostrovsky
Browse files
Add BLiMP
parent
df5d7cf0
Changes
138
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
20 additions
and
0 deletions
+20
-0
tests/tests/testdata/blimp_principle_A_reconstruction-v0-loglikelihood
...estdata/blimp_principle_A_reconstruction-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_principle_A_reconstruction-v0-res.json
...sts/testdata/blimp_principle_A_reconstruction-v0-res.json
+1
-0
tests/tests/testdata/blimp_regular_plural_subject_verb_agreement_1-v0-loglikelihood
..._regular_plural_subject_verb_agreement_1-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_regular_plural_subject_verb_agreement_1-v0-res.json
...blimp_regular_plural_subject_verb_agreement_1-v0-res.json
+1
-0
tests/tests/testdata/blimp_regular_plural_subject_verb_agreement_2-v0-loglikelihood
..._regular_plural_subject_verb_agreement_2-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_regular_plural_subject_verb_agreement_2-v0-res.json
...blimp_regular_plural_subject_verb_agreement_2-v0-res.json
+1
-0
tests/tests/testdata/blimp_sentential_negation_npi_licensor_present-v0-loglikelihood
...sentential_negation_npi_licensor_present-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_sentential_negation_npi_licensor_present-v0-res.json
...limp_sentential_negation_npi_licensor_present-v0-res.json
+1
-0
tests/tests/testdata/blimp_sentential_negation_npi_scope-v0-loglikelihood
...data/blimp_sentential_negation_npi_scope-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_sentential_negation_npi_scope-v0-res.json
.../testdata/blimp_sentential_negation_npi_scope-v0-res.json
+1
-0
tests/tests/testdata/blimp_sentential_subject_island-v0-loglikelihood
...testdata/blimp_sentential_subject_island-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_sentential_subject_island-v0-res.json
...ests/testdata/blimp_sentential_subject_island-v0-res.json
+1
-0
tests/tests/testdata/blimp_superlative_quantifiers_1-v0-loglikelihood
...testdata/blimp_superlative_quantifiers_1-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_superlative_quantifiers_1-v0-res.json
...ests/testdata/blimp_superlative_quantifiers_1-v0-res.json
+1
-0
tests/tests/testdata/blimp_superlative_quantifiers_2-v0-loglikelihood
...testdata/blimp_superlative_quantifiers_2-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_superlative_quantifiers_2-v0-res.json
...ests/testdata/blimp_superlative_quantifiers_2-v0-res.json
+1
-0
tests/tests/testdata/blimp_tough_vs_raising_1-v0-loglikelihood
.../tests/testdata/blimp_tough_vs_raising_1-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_tough_vs_raising_1-v0-res.json
tests/tests/testdata/blimp_tough_vs_raising_1-v0-res.json
+1
-0
tests/tests/testdata/blimp_tough_vs_raising_2-v0-loglikelihood
.../tests/testdata/blimp_tough_vs_raising_2-v0-loglikelihood
+1
-0
tests/tests/testdata/blimp_tough_vs_raising_2-v0-res.json
tests/tests/testdata/blimp_tough_vs_raising_2-v0-res.json
+1
-0
No files found.
tests/tests/testdata/blimp_principle_A_reconstruction-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
894efedfd8750d5b8de6157f9b2ed2b51b5290d3a78ea9b041fc62d34e96efbc
\ No newline at end of file
tests/tests/testdata/blimp_principle_A_reconstruction-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_principle_A_reconstruction"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_principle_A_reconstruction"
:
0
}}
\ No newline at end of file
tests/tests/testdata/blimp_regular_plural_subject_verb_agreement_1-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
5bc0441f31e32443cf761bca6e961d504e1e84b15aa4e1d79e5c8ed5b4c2aa3a
\ No newline at end of file
tests/tests/testdata/blimp_regular_plural_subject_verb_agreement_1-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_regular_plural_subject_verb_agreement_1"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_regular_plural_subject_verb_agreement_1"
:
0
}}
\ No newline at end of file
tests/tests/testdata/blimp_regular_plural_subject_verb_agreement_2-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
f69d9891f59872538962221fccc425b07df7cfbd83cdc546ce83e6b0e9a93f7c
\ No newline at end of file
tests/tests/testdata/blimp_regular_plural_subject_verb_agreement_2-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_regular_plural_subject_verb_agreement_2"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_regular_plural_subject_verb_agreement_2"
:
0
}}
\ No newline at end of file
tests/tests/testdata/blimp_sentential_negation_npi_licensor_present-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
e6666c5657215ff4bfd646b8ee3ae6df956e71c0be9ab1c287fb1b68291dd0d1
\ No newline at end of file
tests/tests/testdata/blimp_sentential_negation_npi_licensor_present-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_sentential_negation_npi_licensor_present"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_sentential_negation_npi_licensor_present"
:
0
}}
\ No newline at end of file
tests/tests/testdata/blimp_sentential_negation_npi_scope-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
32fcbd0a1c6e664af2751bad552587b5ca3911973b07f4fb2cf0a2acd3de5349
\ No newline at end of file
tests/tests/testdata/blimp_sentential_negation_npi_scope-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_sentential_negation_npi_scope"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_sentential_negation_npi_scope"
:
0
}}
\ No newline at end of file
tests/tests/testdata/blimp_sentential_subject_island-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
80f5f98fad26240de2767fe58c4b18d864df41cbfa76f06c84c3fce9f14f4833
\ No newline at end of file
tests/tests/testdata/blimp_sentential_subject_island-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_sentential_subject_island"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_sentential_subject_island"
:
0
}}
\ No newline at end of file
tests/tests/testdata/blimp_superlative_quantifiers_1-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
8a01f6a5ea87a01c0c9b0c7b3bc4de4711bf0ff050976976651182b9ed34a0d4
\ No newline at end of file
tests/tests/testdata/blimp_superlative_quantifiers_1-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_superlative_quantifiers_1"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_superlative_quantifiers_1"
:
0
}}
\ No newline at end of file
tests/tests/testdata/blimp_superlative_quantifiers_2-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
59c20ff0f632cf42afc74ecc682cf92e5e740417b01e6cf9a610a3bc544d2ea5
\ No newline at end of file
tests/tests/testdata/blimp_superlative_quantifiers_2-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_superlative_quantifiers_2"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_superlative_quantifiers_2"
:
0
}}
\ No newline at end of file
tests/tests/testdata/blimp_tough_vs_raising_1-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
973fe56534fdef1207f0fc08dd09a210304c55f33c6cbb17552754bf54f11c86
\ No newline at end of file
tests/tests/testdata/blimp_tough_vs_raising_1-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_tough_vs_raising_1"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_tough_vs_raising_1"
:
0
}}
\ No newline at end of file
tests/tests/testdata/blimp_tough_vs_raising_2-v0-loglikelihood
0 → 100644
View file @
5f48dfc2
d255a10a34f14d77d9526604a17b0f6747d32f62fc2e3a09e9ab10054535fd45
\ No newline at end of file
tests/tests/testdata/blimp_tough_vs_raising_2-v0-res.json
0 → 100644
View file @
5f48dfc2
{
"results"
:
{
"blimp_tough_vs_raising_2"
:
{
"acc"
:
0.485
,
"acc_stderr"
:
0.0158121796418149
}},
"versions"
:
{
"blimp_tough_vs_raising_2"
:
0
}}
\ No newline at end of file
Prev
1
2
3
4
5
6
7
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment