Skip to content

GitLab

Commit cd0983b8 authored by Yu Shi Jie's avatar Yu Shi Jie
Browse files

mmlu-pro: fixed mislabelling of task (math->chemistry)

parent cd441ab1
Show whitespace changes
Inline Side-by-side
Showing with 11 additions and 12 deletions
lm_eval/tasks/mmlu_pro/default/mmlu_pro_chemistry.yaml
View file @ cd0983b8
"dataset_name": "math" "dataset_name": "chemistry"
"description": "The following are multiple choice questions (with answers) about math.\n\ "description": "The following are multiple choice questions (with answers) about chemistry.\n\
\n" \n"
"tag": "mmlu_pro_stem_tasks" "tag": "mmlu_pro_stem_tasks"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_pro_math" "task": "mmlu_pro_chemistry"
"task_alias": "math" "task_alias": "chemistry"
lm_eval/tasks/mmlu_pro/flan_cot_zeroshot/mmlu_pro_chemistry.yaml
View file @ cd0983b8
"dataset_name": "math" "dataset_name": "math"
"description": "The following are multiple choice questions (with answers) about math.\n\ "description": "The following are multiple choice questions (with answers) about math.\n\
\n" \n"
"group": "mmlu_pro_flan_cot_zeroshot_stem" "tag": "mmlu_pro_flan_cot_zeroshot_stem"
"include": "_mmlu_pro_flan_cot_zeroshot_template_yaml" "include": "_mmlu_pro_flan_cot_zeroshot_template_yaml"
"task": "mmlu_pro_flan_cot_zeroshot_chemistry" "task": "mmlu_pro_flan_cot_zeroshot_chemistry"
lm_eval/tasks/mmlu_pro/generative/mmlu_pro_chemistry.yaml
View file @ cd0983b8
"dataset_name": "math" "dataset_name": "chemistry"
"description": "The following are multiple choice questions (with answers) about math.\n\ "description": "The following are multiple choice questions (with answers) about chemitry.\n\
\n" \n"
"group": "mmlu_pro_stem_generative" "tag": "mmlu_pro_stem_generative"
"group_alias": "stem"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_pro_math_generative" "task": "mmlu_pro_chemistry_generative"
"task_alias": "math" "task_alias": "chemistry"
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment