Merge branch 'master' of https://github.com/EleutherAI/lm-evaluation-harness into task-guide

4d147bdd · Jonathan Tow · 011cc891 · dc937d4b · 4d147bdd · 4d147bdd
Commit 4d147bdd authored Sep 17, 2021 by Jonathan Tow
20 changed files
--- a/tests/testdata/arithmetic_3da-v0-res.json
+++ b/tests/testdata/arithmetic_3da-v0-res.json
--- a/tests/testdata/arithmetic_3ds-v0-loglikelihood
+++ b/tests/testdata/arithmetic_3ds-v0-loglikelihood
--- a/tests/testdata/arithmetic_3ds-v0-res.json
+++ b/tests/testdata/arithmetic_3ds-v0-res.json
--- a/tests/testdata/arithmetic_4da-v0-loglikelihood
+++ b/tests/testdata/arithmetic_4da-v0-loglikelihood
--- a/tests/testdata/arithmetic_4da-v0-res.json
+++ b/tests/testdata/arithmetic_4da-v0-res.json
--- a/tests/testdata/arithmetic_4ds-v0-loglikelihood
+++ b/tests/testdata/arithmetic_4ds-v0-loglikelihood
--- a/tests/testdata/arithmetic_4ds-v0-res.json
+++ b/tests/testdata/arithmetic_4ds-v0-res.json
--- a/tests/testdata/arithmetic_5da-v0-loglikelihood
+++ b/tests/testdata/arithmetic_5da-v0-loglikelihood
--- a/tests/testdata/arithmetic_5da-v0-res.json
+++ b/tests/testdata/arithmetic_5da-v0-res.json
--- a/tests/testdata/arithmetic_5ds-v0-loglikelihood
+++ b/tests/testdata/arithmetic_5ds-v0-loglikelihood
--- a/tests/testdata/arithmetic_5ds-v0-res.json
+++ b/tests/testdata/arithmetic_5ds-v0-res.json
--- a/tests/testdata/boolq-v0-loglikelihood
+++ b/tests/testdata/boolq-v0-loglikelihood
--- a/tests/testdata/boolq-v0-res.json
+++ b/tests/testdata/boolq-v0-res.json
--- a/tests/testdata/cb-v0-loglikelihood
+++ b/tests/testdata/cb-v0-loglikelihood
--- a/tests/testdata/cb-v0-res.json
+++ b/tests/testdata/cb-v0-res.json
--- a/tests/testdata/cola-v0-loglikelihood
+++ b/tests/testdata/cola-v0-loglikelihood
--- a/tests/testdata/cola-v0-res.json
+++ b/tests/testdata/cola-v0-res.json
--- a/tests/testdata/copa-v0-loglikelihood
+++ b/tests/testdata/copa-v0-loglikelihood
--- a/tests/testdata/copa-v0-res.json
+++ b/tests/testdata/copa-v0-res.json
--- a/tests/testdata/coqa-v0-greedy_until
+++ b/tests/testdata/coqa-v0-greedy_until