Merge pull request #816 from EleutherAI/flan-benchmark

[Refactor] Flan benchmark

Merge pull request #816 from EleutherAI/flan-benchmark
[Refactor] Flan benchmark
6769119f · Hailey Schoelkopf · GitHub · 4824a832 · 7d5e511c · 6769119f
Unverified Commit 6769119f authored Oct 06, 2023 by Hailey Schoelkopf Committed by GitHub Oct 06, 2023
20 changed files
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_global_facts.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_global_facts.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_biology.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_biology.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_chemistry.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_chemistry.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_computer_science.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_computer_science.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_european_history.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_european_history.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_geography.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_geography.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_government_and_politics.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_government_and_politics.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_macroeconomics.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_macroeconomics.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_mathematics.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_mathematics.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_microeconomics.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_microeconomics.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_physics.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_physics.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_psychology.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_psychology.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_statistics.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_statistics.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_us_history.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_us_history.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_world_history.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_high_school_world_history.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_human_aging.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_human_aging.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_human_sexuality.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_human_sexuality.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_international_law.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_international_law.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_jurisprudence.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_jurisprudence.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_logical_fallacies.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/mmlu_gen_logical_fallacies.yaml