separate category for `global_mmlu` (#2652)

* separate category * set version 0.0 * apply precommit

separate category for `global_mmlu` (#2652)
* separate category * set version 0.0 * apply precommit
5c006ed4 · Minho Ryu · GitHub · 370e2f9e · 5c006ed4 · 5c006ed4
Unverified Commit 5c006ed4 authored Jan 25, 2025 by Minho Ryu Committed by GitHub Jan 24, 2025
20 changed files
--- a/lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_stem.yaml
+++ b/lm_eval/tasks/global_mmlu/default/yo/global_mmlu_yo_stem.yaml
--- a/lm_eval/tasks/global_mmlu/default/yo/utils.py
+++ b/lm_eval/tasks/global_mmlu/default/yo/utils.py
--- a/lm_eval/tasks/global_mmlu/default/zh/_global_mmlu_zh.yaml
+++ b/lm_eval/tasks/global_mmlu/default/zh/_global_mmlu_zh.yaml
--- a/lm_eval/tasks/global_mmlu/default/zh/_zh_template_yaml
+++ b/lm_eval/tasks/global_mmlu/default/zh/_zh_template_yaml
--- a/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_business.yaml
+++ b/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_business.yaml
--- a/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_humanities.yaml
+++ b/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_humanities.yaml
--- a/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_medical.yaml
+++ b/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_medical.yaml
--- a/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_other.yaml
+++ b/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_other.yaml
--- a/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_social_sciences.yaml
+++ b/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_social_sciences.yaml
--- a/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_stem.yaml
+++ b/lm_eval/tasks/global_mmlu/default/zh/global_mmlu_zh_stem.yaml
--- a/lm_eval/tasks/global_mmlu/default/zh/utils.py
+++ b/lm_eval/tasks/global_mmlu/default/zh/utils.py
--- a/lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am.yaml
+++ b/lm_eval/tasks/global_mmlu/full/am/_global_mmlu_full_am.yaml
--- a/lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar.yaml
+++ b/lm_eval/tasks/global_mmlu/full/ar/_global_mmlu_full_ar.yaml
--- a/lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn.yaml
+++ b/lm_eval/tasks/global_mmlu/full/bn/_global_mmlu_full_bn.yaml
--- a/lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs.yaml
+++ b/lm_eval/tasks/global_mmlu/full/cs/_global_mmlu_full_cs.yaml
--- a/lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de.yaml
+++ b/lm_eval/tasks/global_mmlu/full/de/_global_mmlu_full_de.yaml
--- a/lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el.yaml
+++ b/lm_eval/tasks/global_mmlu/full/el/_global_mmlu_full_el.yaml
--- a/lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en.yaml
+++ b/lm_eval/tasks/global_mmlu/full/en/_global_mmlu_full_en.yaml
--- a/lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es.yaml
+++ b/lm_eval/tasks/global_mmlu/full/es/_global_mmlu_full_es.yaml
--- a/lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa.yaml
+++ b/lm_eval/tasks/global_mmlu/full/fa/_global_mmlu_full_fa.yaml