Add tests to make sure task definitions don't change without the version also changing

17c825d1 · Leo Gao · 105fa974 · 17c825d1 · 17c825d1 · 17c825d1
Commit 17c825d1 authored Jun 04, 2021 by Leo Gao
20 changed files
--- a/tests/testdata/hendrycksTest-global_facts-v0-loglikelihood.json
+++ b/tests/testdata/hendrycksTest-global_facts-v0-loglikelihood.json
--- a/tests/testdata/hendrycksTest-global_facts-v0-res.json
+++ b/tests/testdata/hendrycksTest-global_facts-v0-res.json
--- a/tests/testdata/hendrycksTest-high_school_biology-v0-loglikelihood.json
+++ b/tests/testdata/hendrycksTest-high_school_biology-v0-loglikelihood.json
--- a/tests/testdata/hendrycksTest-high_school_biology-v0-res.json
+++ b/tests/testdata/hendrycksTest-high_school_biology-v0-res.json
--- a/tests/testdata/hendrycksTest-high_school_chemistry-v0-loglikelihood.json
+++ b/tests/testdata/hendrycksTest-high_school_chemistry-v0-loglikelihood.json
--- a/tests/testdata/hendrycksTest-high_school_chemistry-v0-res.json
+++ b/tests/testdata/hendrycksTest-high_school_chemistry-v0-res.json
--- a/tests/testdata/hendrycksTest-high_school_computer_science-v0-loglikelihood.json
+++ b/tests/testdata/hendrycksTest-high_school_computer_science-v0-loglikelihood.json
--- a/tests/testdata/hendrycksTest-high_school_computer_science-v0-res.json
+++ b/tests/testdata/hendrycksTest-high_school_computer_science-v0-res.json
--- a/tests/testdata/hendrycksTest-high_school_european_history-v0-loglikelihood.json
+++ b/tests/testdata/hendrycksTest-high_school_european_history-v0-loglikelihood.json
--- a/tests/testdata/hendrycksTest-high_school_european_history-v0-res.json
+++ b/tests/testdata/hendrycksTest-high_school_european_history-v0-res.json
--- a/tests/testdata/hendrycksTest-high_school_geography-v0-loglikelihood.json
+++ b/tests/testdata/hendrycksTest-high_school_geography-v0-loglikelihood.json
--- a/tests/testdata/hendrycksTest-high_school_geography-v0-res.json
+++ b/tests/testdata/hendrycksTest-high_school_geography-v0-res.json
--- a/tests/testdata/hendrycksTest-high_school_government_and_politics-v0-loglikelihood.json
+++ b/tests/testdata/hendrycksTest-high_school_government_and_politics-v0-loglikelihood.json
--- a/tests/testdata/hendrycksTest-high_school_government_and_politics-v0-res.json
+++ b/tests/testdata/hendrycksTest-high_school_government_and_politics-v0-res.json
--- a/tests/testdata/hendrycksTest-high_school_macroeconomics-v0-loglikelihood.json
+++ b/tests/testdata/hendrycksTest-high_school_macroeconomics-v0-loglikelihood.json
--- a/tests/testdata/hendrycksTest-high_school_macroeconomics-v0-res.json
+++ b/tests/testdata/hendrycksTest-high_school_macroeconomics-v0-res.json
--- a/tests/testdata/hendrycksTest-high_school_mathematics-v0-loglikelihood.json
+++ b/tests/testdata/hendrycksTest-high_school_mathematics-v0-loglikelihood.json
--- a/tests/testdata/hendrycksTest-high_school_mathematics-v0-res.json
+++ b/tests/testdata/hendrycksTest-high_school_mathematics-v0-res.json
--- a/tests/testdata/hendrycksTest-high_school_microeconomics-v0-loglikelihood.json
+++ b/tests/testdata/hendrycksTest-high_school_microeconomics-v0-loglikelihood.json
--- a/tests/testdata/hendrycksTest-high_school_microeconomics-v0-res.json
+++ b/tests/testdata/hendrycksTest-high_school_microeconomics-v0-res.json