Multilingual MMLU for Llama instruct models (#2826)

* Multilingual MMLU * Refactor process_docs function calls for clarity and consistency

Multilingual MMLU for Llama instruct models (#2826)
* Multilingual MMLU * Refactor process_docs function calls for clarity and consistency
1b357a68 · Alexandre Marques · GitHub · 1afb190c · 1b357a68 · 1b357a68
Unverified Commit 1b357a68 authored Mar 25, 2025 by Alexandre Marques Committed by GitHub Mar 25, 2025
20 changed files
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_chemistry.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_chemistry.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_computer_science.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_computer_science.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_mathematics.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_mathematics.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_medicine.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_medicine.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_physics.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_college_physics.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_computer_security.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_computer_security.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_conceptual_physics.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_conceptual_physics.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_econometrics.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_econometrics.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_electrical_engineering.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_electrical_engineering.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_elementary_mathematics.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_elementary_mathematics.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_formal_logic.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_formal_logic.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_global_facts.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_global_facts.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_biology.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_biology.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_chemistry.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_chemistry.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_computer_science.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_computer_science.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_european_history.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_european_history.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_geography.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_geography.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_government_and_politics.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_government_and_politics.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_macroeconomics.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_macroeconomics.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_mathematics.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_fr/mmlu_fr_high_school_mathematics.yaml