Merge pull request #816 from EleutherAI/flan-benchmark

[Refactor] Flan benchmark

Merge pull request #816 from EleutherAI/flan-benchmark
[Refactor] Flan benchmark
6769119f · Hailey Schoelkopf · GitHub · 4824a832 · 7d5e511c · 6769119f
Unverified Commit 6769119f authored Oct 06, 2023 by Hailey Schoelkopf Committed by GitHub Oct 06, 2023
20 changed files
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_logical_fallacies.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_logical_fallacies.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_machine_learning.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_machine_learning.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_management.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_management.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_marketing.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_marketing.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_medical_genetics.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_medical_genetics.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_miscellaneous.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_miscellaneous.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_disputes.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_disputes.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_scenarios.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_moral_scenarios.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_nutrition.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_nutrition.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_philosophy.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_philosophy.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_prehistory.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_prehistory.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_accounting.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_accounting.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_law.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_law.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_medicine.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_medicine.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_psychology.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_psychology.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_public_relations.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_public_relations.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_security_studies.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_security_studies.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_sociology.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_sociology.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_us_foreign_policy.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_us_foreign_policy.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_virology.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_virology.yaml