lm_eval/tasks/mmlu/continuation/mmlu_astronomy.yaml · 5a7ed3eee00eca378f157abbe9baf4d406f063f2 · gaoqiong / lm-evaluation-harness

"vllm_flash_attn/ops/triton/mlp.py" did not exist on "b630aef53fb263520a10f86fe08cd5cfb0360702"

add task for mmlu evaluation in arc multiple choice format (#1745) · 9097ad3e

jonabur authored May 08, 2024



* add mmlu arc style evaluation

* rename arc_style to continuation

---------
Co-authored-by: Jonathan Burdge <jburdge@mahti-login11.mahti.csc.fi>
Co-authored-by: Jonathan Burdge <jburdge@mahti-login12.mahti.csc.fi>

9097ad3e

mmlu_astronomy.yaml 226 Bytes

Replace mmlu_astronomy.yaml