separate category for `global_mmlu` (#2652)

* separate category * set version 0.0 * apply precommit

separate category for `global_mmlu` (#2652)
* separate category * set version 0.0 * apply precommit
5c006ed4 · Minho Ryu · GitHub · 370e2f9e · 5c006ed4 · 5c006ed4
Unverified Commit 5c006ed4 authored Jan 25, 2025 by Minho Ryu Committed by GitHub Jan 24, 2025
20 changed files
--- a/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_humanities.yaml
+++ b/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_humanities.yaml
--- a/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_medical.yaml
+++ b/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_medical.yaml
--- a/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_other.yaml
+++ b/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_other.yaml
--- a/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_social_sciences.yaml
+++ b/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_social_sciences.yaml
--- a/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_stem.yaml
+++ b/lm_eval/tasks/global_mmlu/default/ja/global_mmlu_ja_stem.yaml
--- a/lm_eval/tasks/global_mmlu/default/ja/utils.py
+++ b/lm_eval/tasks/global_mmlu/default/ja/utils.py
--- a/lm_eval/tasks/global_mmlu/default/ko/_global_mmlu_ko.yaml
+++ b/lm_eval/tasks/global_mmlu/default/ko/_global_mmlu_ko.yaml
--- a/lm_eval/tasks/global_mmlu/default/ko/_ko_template_yaml
+++ b/lm_eval/tasks/global_mmlu/default/ko/_ko_template_yaml
--- a/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_business.yaml
+++ b/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_business.yaml
--- a/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_humanities.yaml
+++ b/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_humanities.yaml
--- a/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_medical.yaml
+++ b/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_medical.yaml
--- a/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_other.yaml
+++ b/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_other.yaml
--- a/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_social_sciences.yaml
+++ b/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_social_sciences.yaml
--- a/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_stem.yaml
+++ b/lm_eval/tasks/global_mmlu/default/ko/global_mmlu_ko_stem.yaml
--- a/lm_eval/tasks/global_mmlu/default/ko/utils.py
+++ b/lm_eval/tasks/global_mmlu/default/ko/utils.py
--- a/lm_eval/tasks/global_mmlu/default/pt/_global_mmlu_pt.yaml
+++ b/lm_eval/tasks/global_mmlu/default/pt/_global_mmlu_pt.yaml
--- a/lm_eval/tasks/global_mmlu/default/pt/_pt_template_yaml
+++ b/lm_eval/tasks/global_mmlu/default/pt/_pt_template_yaml
--- a/lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_business.yaml
+++ b/lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_business.yaml
--- a/lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_humanities.yaml
+++ b/lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_humanities.yaml
--- a/lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_medical.yaml
+++ b/lm_eval/tasks/global_mmlu/default/pt/global_mmlu_pt_medical.yaml