Multilingual MMLU for Llama instruct models (#2826)

* Multilingual MMLU * Refactor process_docs function calls for clarity and consistency

Multilingual MMLU for Llama instruct models (#2826)
* Multilingual MMLU * Refactor process_docs function calls for clarity and consistency
1b357a68 · Alexandre Marques · GitHub · 1afb190c · 1b357a68 · 1b357a68
Unverified Commit 1b357a68 authored Mar 25, 2025 by Alexandre Marques Committed by GitHub Mar 25, 2025
8 changed files
--- a/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_psychology.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_professional_psychology.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_public_relations.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_public_relations.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_security_studies.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_security_studies.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_sociology.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_sociology.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_us_foreign_policy.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_us_foreign_policy.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_virology.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_virology.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_world_religions.yaml
+++ b/lm_eval/tasks/llama3/instruct/mmlu_th/mmlu_th_world_religions.yaml
--- a/lm_eval/tasks/llama3/instruct/mmlu_th/utils.py
+++ b/lm_eval/tasks/llama3/instruct/mmlu_th/utils.py