Merge branch 'group-agg-rework' of...

Merge branch 'group-agg-rework' of https://github.com/EleutherAI/lm-evaluation-harness into multiprompt

Merge branch 'group-agg-rework' of...
Merge branch 'group-agg-rework' of https://github.com/EleutherAI/lm-evaluation-harness into multiprompt
88486e57 · lintangsutawika · 5971f2ca · ba73d131 · 88486e57 · 88486e57
Commit 88486e57 authored Jul 05, 2024 by lintangsutawika
20 changed files
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_accounting.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_accounting.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_law.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_law.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_medicine.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_medicine.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_psychology.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_professional_psychology.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_public_relations.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_public_relations.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_security_studies.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_security_studies.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_sociology.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_sociology.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_us_foreign_policy.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_us_foreign_policy.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_virology.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_virology.yaml
--- a/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_world_religions.yaml
+++ b/lm_eval/tasks/mmlu/flan_cot_zeroshot/mmlu_world_religions.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu_flan_generative_template_yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/_mmlu_flan_generative_template_yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_abstract_algebra.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_abstract_algebra.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_anatomy.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_anatomy.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_astronomy.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_astronomy.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_business_ethics.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_business_ethics.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_clinical_knowledge.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_clinical_knowledge.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_biology.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_biology.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_chemistry.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_chemistry.yaml
--- a/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_computer_science.yaml
+++ b/lm_eval/tasks/mmlu/flan_n_shot/generative/mmlu_college_computer_science.yaml