Merge pull request #922 from EleutherAI/mmlu_subgroups

[Refactor] Mmlu subgroups and weight avg

Merge pull request #922 from EleutherAI/mmlu_subgroups
[Refactor] Mmlu subgroups and weight avg
815f59e6 · Lintang Sutawika · GitHub · 3533e4b9 · 44124d95 · 815f59e6
Unverified Commit 815f59e6 authored Nov 06, 2023 by Lintang Sutawika Committed by GitHub Nov 06, 2023
20 changed files
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_jurisprudence.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_jurisprudence.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_logical_fallacies.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_logical_fallacies.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_machine_learning.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_machine_learning.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_management.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_management.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_marketing.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_marketing.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_medical_genetics.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_medical_genetics.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_miscellaneous.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_miscellaneous.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_disputes.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_disputes.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_scenarios.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_moral_scenarios.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_nutrition.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_nutrition.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_philosophy.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_philosophy.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_prehistory.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_prehistory.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_accounting.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_accounting.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_law.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_law.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_medicine.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_medicine.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_psychology.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_professional_psychology.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_public_relations.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_public_relations.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_security_studies.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_security_studies.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_sociology.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_sociology.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_us_foreign_policy.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_us_foreign_policy.yaml