Multilingual MMLU for Llama instruct models (#2826)

* Multilingual MMLU * Refactor process_docs function calls for clarity and consistency

Multilingual MMLU for Llama instruct models (#2826)
* Multilingual MMLU * Refactor process_docs function calls for clarity and consistency
1b357a68 · Alexandre Marques · GitHub · 1afb190c · 1b357a68 · 1b357a68
Unverified Commit 1b357a68 authored Mar 25, 2025 by Alexandre Marques Committed by GitHub Mar 25, 2025
20 changed files
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_us_history.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_us_history.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_world_history.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_high_school_world_history.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_human_aging.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_human_aging.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_human_sexuality.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_human_sexuality.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_international_law.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_international_law.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_jurisprudence.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_jurisprudence.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_logical_fallacies.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_logical_fallacies.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_machine_learning.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_machine_learning.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_management.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_management.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_marketing.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_marketing.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_medical_genetics.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_medical_genetics.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_miscellaneous.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_miscellaneous.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_moral_disputes.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_moral_disputes.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_moral_scenarios.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_moral_scenarios.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_nutrition.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_nutrition.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_philosophy.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_philosophy.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_prehistory.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_prehistory.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_accounting.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_accounting.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_law.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_law.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_medicine.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_es/mmlu_es_professional_medicine.yaml