Merge branch 'master' of github.com:EleutherAI/lm-evaluation-harness into...

Merge branch 'master' of github.com:EleutherAI/lm-evaluation-harness into evaluator-description-option

Merge branch 'master' of github.com:EleutherAI/lm-evaluation-harness into...
Merge branch 'master' of github.com:EleutherAI/lm-evaluation-harness into evaluator-description-option
5792030c · Leo Gao · b2f6bce8 · 8728710c · 5792030c · 5792030c
Commit 5792030c authored Jan 08, 2022 by Leo Gao
20 changed files
--- a/tests/tests/testdata/blimp_principle_A_domain_3-v0-loglikelihood
+++ b/tests/tests/testdata/blimp_principle_A_domain_3-v0-loglikelihood
--- a/tests/tests/testdata/blimp_principle_A_domain_3-v0-res.json
+++ b/tests/tests/testdata/blimp_principle_A_domain_3-v0-res.json
--- a/tests/tests/testdata/blimp_principle_A_reconstruction-v0-loglikelihood
+++ b/tests/tests/testdata/blimp_principle_A_reconstruction-v0-loglikelihood
--- a/tests/tests/testdata/blimp_principle_A_reconstruction-v0-res.json
+++ b/tests/tests/testdata/blimp_principle_A_reconstruction-v0-res.json
--- a/tests/tests/testdata/blimp_regular_plural_subject_verb_agreement_1-v0-loglikelihood
+++ b/tests/tests/testdata/blimp_regular_plural_subject_verb_agreement_1-v0-loglikelihood
--- a/tests/tests/testdata/blimp_regular_plural_subject_verb_agreement_1-v0-res.json
+++ b/tests/tests/testdata/blimp_regular_plural_subject_verb_agreement_1-v0-res.json
--- a/tests/tests/testdata/blimp_regular_plural_subject_verb_agreement_2-v0-loglikelihood
+++ b/tests/tests/testdata/blimp_regular_plural_subject_verb_agreement_2-v0-loglikelihood
--- a/tests/tests/testdata/blimp_regular_plural_subject_verb_agreement_2-v0-res.json
+++ b/tests/tests/testdata/blimp_regular_plural_subject_verb_agreement_2-v0-res.json
--- a/tests/tests/testdata/blimp_sentential_negation_npi_licensor_present-v0-loglikelihood
+++ b/tests/tests/testdata/blimp_sentential_negation_npi_licensor_present-v0-loglikelihood
--- a/tests/tests/testdata/blimp_sentential_negation_npi_licensor_present-v0-res.json
+++ b/tests/tests/testdata/blimp_sentential_negation_npi_licensor_present-v0-res.json
--- a/tests/tests/testdata/blimp_sentential_negation_npi_scope-v0-loglikelihood
+++ b/tests/tests/testdata/blimp_sentential_negation_npi_scope-v0-loglikelihood
--- a/tests/tests/testdata/blimp_sentential_negation_npi_scope-v0-res.json
+++ b/tests/tests/testdata/blimp_sentential_negation_npi_scope-v0-res.json
--- a/tests/tests/testdata/blimp_sentential_subject_island-v0-loglikelihood
+++ b/tests/tests/testdata/blimp_sentential_subject_island-v0-loglikelihood
--- a/tests/tests/testdata/blimp_sentential_subject_island-v0-res.json
+++ b/tests/tests/testdata/blimp_sentential_subject_island-v0-res.json
--- a/tests/tests/testdata/blimp_superlative_quantifiers_1-v0-loglikelihood
+++ b/tests/tests/testdata/blimp_superlative_quantifiers_1-v0-loglikelihood
--- a/tests/tests/testdata/blimp_superlative_quantifiers_1-v0-res.json
+++ b/tests/tests/testdata/blimp_superlative_quantifiers_1-v0-res.json
--- a/tests/tests/testdata/blimp_superlative_quantifiers_2-v0-loglikelihood
+++ b/tests/tests/testdata/blimp_superlative_quantifiers_2-v0-loglikelihood
--- a/tests/tests/testdata/blimp_superlative_quantifiers_2-v0-res.json
+++ b/tests/tests/testdata/blimp_superlative_quantifiers_2-v0-res.json
--- a/tests/tests/testdata/blimp_tough_vs_raising_1-v0-loglikelihood
+++ b/tests/tests/testdata/blimp_tough_vs_raising_1-v0-loglikelihood
--- a/tests/tests/testdata/blimp_tough_vs_raising_1-v0-res.json
+++ b/tests/tests/testdata/blimp_tough_vs_raising_1-v0-res.json