Merge branch 'group-agg-rework' of...

Merge branch 'group-agg-rework' of https://github.com/EleutherAI/lm-evaluation-harness into multiprompt

Merge branch 'group-agg-rework' of...
Merge branch 'group-agg-rework' of https://github.com/EleutherAI/lm-evaluation-harness into multiprompt
88486e57 · lintangsutawika · 5971f2ca · ba73d131 · 88486e57 · 88486e57
Commit 88486e57 authored Jul 05, 2024 by lintangsutawika
20 changed files
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_psychology.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_psychology.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_statistics.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_statistics.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_us_history.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_us_history.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_world_history.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_high_school_world_history.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_aging.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_aging.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_sexuality.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_human_sexuality.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_international_law.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_international_law.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_jurisprudence.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_jurisprudence.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_logical_fallacies.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_logical_fallacies.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_machine_learning.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_machine_learning.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_management.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_management.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_marketing.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_marketing.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_medical_genetics.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_medical_genetics.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_miscellaneous.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_miscellaneous.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_disputes.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_disputes.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_scenarios.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_moral_scenarios.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_nutrition.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_nutrition.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_philosophy.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_philosophy.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_prehistory.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_prehistory.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_accounting.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_professional_accounting.yaml