lm_eval/tasks/mmlu/continuation/mmlu_human_sexuality.yaml · 3e1301bb10e6b20b3b2d399e98794cca0e44f6f9 · gaoqiong / lm-evaluation-harness

"...training_service/pai/paiK8S/paiK8STrainingService.ts" did not exist on "6de15707c1f9ec7409432aea231e149c91b79626"

add task for mmlu evaluation in arc multiple choice format (#1745) · 9097ad3e

jonabur authored May 08, 2024



* add mmlu arc style evaluation

* rename arc_style to continuation

---------
Co-authored-by: Jonathan Burdge <jburdge@mahti-login11.mahti.csc.fi>
Co-authored-by: Jonathan Burdge <jburdge@mahti-login12.mahti.csc.fi>

9097ad3e

mmlu_human_sexuality.yaml 256 Bytes

Replace mmlu_human_sexuality.yaml