Add new dataset MMLU-SR tasks (#2032)

* add mmlusr tasks * renamed all tasks names in mmlusr * edit format and readme * added mmlu_sr * mmlu_sr -> mmlusr * update --------- Co-authored-by: lintangsutawika <lintang@eleuther.ai>

Add new dataset MMLU-SR tasks (#2032)
* add mmlusr tasks * renamed all tasks names in mmlusr * edit format and readme * added mmlu_sr * mmlu_sr -> mmlusr * update --------- Co-authored-by: lintangsutawika <lintang@eleuther.ai>
d5f39bf8 · SuperCat · GitHub · cdd954f9 · d5f39bf8 · d5f39bf8
Unverified Commit d5f39bf8 authored Jul 12, 2024 by SuperCat Committed by GitHub Jul 12, 2024
20 changed files
--- a/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_virology.yaml
+++ b/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_virology.yaml
--- a/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_world_religions.yaml
+++ b/lm_eval/tasks/mmlusr/question_and_answer/question_and_answer_world_religions.yaml
--- a/lm_eval/tasks/mmlusr/question_and_answer/utils.py
+++ b/lm_eval/tasks/mmlusr/question_and_answer/utils.py
--- a/lm_eval/tasks/mmlusr/question_only/_mmlusr_q_yml
+++ b/lm_eval/tasks/mmlusr/question_only/_mmlusr_q_yml
--- a/lm_eval/tasks/mmlusr/question_only/_question_only.yaml
+++ b/lm_eval/tasks/mmlusr/question_only/_question_only.yaml
--- a/lm_eval/tasks/mmlusr/question_only/question_only_abstract_algebra.yaml
+++ b/lm_eval/tasks/mmlusr/question_only/question_only_abstract_algebra.yaml
--- a/lm_eval/tasks/mmlusr/question_only/question_only_anatomy.yaml
+++ b/lm_eval/tasks/mmlusr/question_only/question_only_anatomy.yaml
--- a/lm_eval/tasks/mmlusr/question_only/question_only_astronomy.yaml
+++ b/lm_eval/tasks/mmlusr/question_only/question_only_astronomy.yaml
--- a/lm_eval/tasks/mmlusr/question_only/question_only_business_ethics.yaml
+++ b/lm_eval/tasks/mmlusr/question_only/question_only_business_ethics.yaml
--- a/lm_eval/tasks/mmlusr/question_only/question_only_clinical_knowledge.yaml
+++ b/lm_eval/tasks/mmlusr/question_only/question_only_clinical_knowledge.yaml
--- a/lm_eval/tasks/mmlusr/question_only/question_only_college_biology.yaml
+++ b/lm_eval/tasks/mmlusr/question_only/question_only_college_biology.yaml
--- a/lm_eval/tasks/mmlusr/question_only/question_only_college_chemistry.yaml
+++ b/lm_eval/tasks/mmlusr/question_only/question_only_college_chemistry.yaml
--- a/lm_eval/tasks/mmlusr/question_only/question_only_college_computer_science.yaml
+++ b/lm_eval/tasks/mmlusr/question_only/question_only_college_computer_science.yaml
--- a/lm_eval/tasks/mmlusr/question_only/question_only_college_mathematics.yaml
+++ b/lm_eval/tasks/mmlusr/question_only/question_only_college_mathematics.yaml
--- a/lm_eval/tasks/mmlusr/question_only/question_only_college_medicine.yaml
+++ b/lm_eval/tasks/mmlusr/question_only/question_only_college_medicine.yaml
--- a/lm_eval/tasks/mmlusr/question_only/question_only_college_physics.yaml
+++ b/lm_eval/tasks/mmlusr/question_only/question_only_college_physics.yaml
--- a/lm_eval/tasks/mmlusr/question_only/question_only_computer_security.yaml
+++ b/lm_eval/tasks/mmlusr/question_only/question_only_computer_security.yaml
--- a/lm_eval/tasks/mmlusr/question_only/question_only_conceptual_physics.yaml
+++ b/lm_eval/tasks/mmlusr/question_only/question_only_conceptual_physics.yaml
--- a/lm_eval/tasks/mmlusr/question_only/question_only_econometrics.yaml
+++ b/lm_eval/tasks/mmlusr/question_only/question_only_econometrics.yaml
--- a/lm_eval/tasks/mmlusr/question_only/question_only_electrical_engineering.yaml
+++ b/lm_eval/tasks/mmlusr/question_only/question_only_electrical_engineering.yaml