Commit e200c24e authored by lintangsutawika's avatar lintangsutawika
Browse files

update mmlu

parent 43765669
"dataset_name": "medical_genetics" "dataset_name": "medical_genetics"
"description": "The following are multiple choice questions (with answers) about medical\ "description": "The following are multiple choice questions (with answers) about medical\
\ genetics.\n\n" \ genetics.\n\n"
"group": "mmlu_other_tasks" "tag": "mmlu_other_tasks"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_medical_genetics" "task": "mmlu_medical_genetics"
"task_alias": "medical_genetics" "task_alias": "medical_genetics"
"dataset_name": "miscellaneous" "dataset_name": "miscellaneous"
"description": "The following are multiple choice questions (with answers) about miscellaneous.\n\ "description": "The following are multiple choice questions (with answers) about miscellaneous.\n\
\n" \n"
"group": "mmlu_other_tasks" "tag": "mmlu_other_tasks"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_miscellaneous" "task": "mmlu_miscellaneous"
"task_alias": "miscellaneous" "task_alias": "miscellaneous"
"dataset_name": "moral_disputes" "dataset_name": "moral_disputes"
"description": "The following are multiple choice questions (with answers) about moral\ "description": "The following are multiple choice questions (with answers) about moral\
\ disputes.\n\n" \ disputes.\n\n"
"group": "mmlu_humanities_tasks" "tag": "mmlu_humanities_tasks"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_moral_disputes" "task": "mmlu_moral_disputes"
"task_alias": "moral_disputes" "task_alias": "moral_disputes"
"dataset_name": "moral_scenarios" "dataset_name": "moral_scenarios"
"description": "The following are multiple choice questions (with answers) about moral\ "description": "The following are multiple choice questions (with answers) about moral\
\ scenarios.\n\n" \ scenarios.\n\n"
"group": "mmlu_humanities_tasks" "tag": "mmlu_humanities_tasks"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_moral_scenarios" "task": "mmlu_moral_scenarios"
"task_alias": "moral_scenarios" "task_alias": "moral_scenarios"
"dataset_name": "nutrition" "dataset_name": "nutrition"
"description": "The following are multiple choice questions (with answers) about nutrition.\n\ "description": "The following are multiple choice questions (with answers) about nutrition.\n\
\n" \n"
"group": "mmlu_other_tasks" "tag": "mmlu_other_tasks"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_nutrition" "task": "mmlu_nutrition"
"task_alias": "nutrition" "task_alias": "nutrition"
"dataset_name": "philosophy" "dataset_name": "philosophy"
"description": "The following are multiple choice questions (with answers) about philosophy.\n\ "description": "The following are multiple choice questions (with answers) about philosophy.\n\
\n" \n"
"group": "mmlu_humanities_tasks" "tag": "mmlu_humanities_tasks"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_philosophy" "task": "mmlu_philosophy"
"task_alias": "philosophy" "task_alias": "philosophy"
"dataset_name": "prehistory" "dataset_name": "prehistory"
"description": "The following are multiple choice questions (with answers) about prehistory.\n\ "description": "The following are multiple choice questions (with answers) about prehistory.\n\
\n" \n"
"group": "mmlu_humanities_tasks" "tag": "mmlu_humanities_tasks"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_prehistory" "task": "mmlu_prehistory"
"task_alias": "prehistory" "task_alias": "prehistory"
"dataset_name": "professional_accounting" "dataset_name": "professional_accounting"
"description": "The following are multiple choice questions (with answers) about professional\ "description": "The following are multiple choice questions (with answers) about professional\
\ accounting.\n\n" \ accounting.\n\n"
"group": "mmlu_other_tasks" "tag": "mmlu_other_tasks"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_professional_accounting" "task": "mmlu_professional_accounting"
"task_alias": "professional_accounting" "task_alias": "professional_accounting"
"dataset_name": "professional_law" "dataset_name": "professional_law"
"description": "The following are multiple choice questions (with answers) about professional\ "description": "The following are multiple choice questions (with answers) about professional\
\ law.\n\n" \ law.\n\n"
"group": "mmlu_humanities_tasks" "tag": "mmlu_humanities_tasks"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_professional_law" "task": "mmlu_professional_law"
"task_alias": "professional_law" "task_alias": "professional_law"
"dataset_name": "professional_medicine" "dataset_name": "professional_medicine"
"description": "The following are multiple choice questions (with answers) about professional\ "description": "The following are multiple choice questions (with answers) about professional\
\ medicine.\n\n" \ medicine.\n\n"
"group": "mmlu_other_tasks" "tag": "mmlu_other_tasks"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_professional_medicine" "task": "mmlu_professional_medicine"
"task_alias": "professional_medicine" "task_alias": "professional_medicine"
"dataset_name": "professional_psychology" "dataset_name": "professional_psychology"
"description": "The following are multiple choice questions (with answers) about professional\ "description": "The following are multiple choice questions (with answers) about professional\
\ psychology.\n\n" \ psychology.\n\n"
"group": "mmlu_social_sciences_tasks" "tag": "mmlu_social_sciences_tasks"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_professional_psychology" "task": "mmlu_professional_psychology"
"task_alias": "professional_psychology" "task_alias": "professional_psychology"
"dataset_name": "public_relations" "dataset_name": "public_relations"
"description": "The following are multiple choice questions (with answers) about public\ "description": "The following are multiple choice questions (with answers) about public\
\ relations.\n\n" \ relations.\n\n"
"group": "mmlu_social_sciences_tasks" "tag": "mmlu_social_sciences_tasks"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_public_relations" "task": "mmlu_public_relations"
"task_alias": "public_relations" "task_alias": "public_relations"
"dataset_name": "security_studies" "dataset_name": "security_studies"
"description": "The following are multiple choice questions (with answers) about security\ "description": "The following are multiple choice questions (with answers) about security\
\ studies.\n\n" \ studies.\n\n"
"group": "mmlu_social_sciences_tasks" "tag": "mmlu_social_sciences_tasks"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_security_studies" "task": "mmlu_security_studies"
"task_alias": "security_studies" "task_alias": "security_studies"
"dataset_name": "sociology" "dataset_name": "sociology"
"description": "The following are multiple choice questions (with answers) about sociology.\n\ "description": "The following are multiple choice questions (with answers) about sociology.\n\
\n" \n"
"group": "mmlu_social_sciences_tasks" "tag": "mmlu_social_sciences_tasks"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_sociology" "task": "mmlu_sociology"
"task_alias": "sociology" "task_alias": "sociology"
"dataset_name": "us_foreign_policy" "dataset_name": "us_foreign_policy"
"description": "The following are multiple choice questions (with answers) about us\ "description": "The following are multiple choice questions (with answers) about us\
\ foreign policy.\n\n" \ foreign policy.\n\n"
"group": "mmlu_social_sciences_tasks" "tag": "mmlu_social_sciences_tasks"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_us_foreign_policy" "task": "mmlu_us_foreign_policy"
"task_alias": "us_foreign_policy" "task_alias": "us_foreign_policy"
"dataset_name": "virology" "dataset_name": "virology"
"description": "The following are multiple choice questions (with answers) about virology.\n\ "description": "The following are multiple choice questions (with answers) about virology.\n\
\n" \n"
"group": "mmlu_other_tasks" "tag": "mmlu_other_tasks"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_virology" "task": "mmlu_virology"
"task_alias": "virology" "task_alias": "virology"
"dataset_name": "world_religions" "dataset_name": "world_religions"
"description": "The following are multiple choice questions (with answers) about world\ "description": "The following are multiple choice questions (with answers) about world\
\ religions.\n\n" \ religions.\n\n"
"group": "mmlu_humanities_tasks" "tag": "mmlu_humanities_tasks"
"include": "_default_template_yaml" "include": "_default_template_yaml"
"task": "mmlu_world_religions" "task": "mmlu_world_religions"
"task_alias": "world_religions" "task_alias": "world_religions"
...@@ -54,6 +54,6 @@ fewshot_config: ...@@ -54,6 +54,6 @@ fewshot_config:
not have any roots. For c = 2 the polynomial x^2 + 2 has two roots at x = 1 not have any roots. For c = 2 the polynomial x^2 + 2 has two roots at x = 1
and x = 2. Hence Z_3[x]/(x^2 + c) is a field if and only if c = 1. The answer and x = 2. Hence Z_3[x]/(x^2 + c) is a field if and only if c = 1. The answer
is (B).' is (B).'
group: mmlu_flan_cot_fewshot_stem tag: mmlu_flan_cot_fewshot_stem
include: _mmlu_flan_cot_fewshot_template_yaml include: _mmlu_flan_cot_fewshot_template_yaml
task: mmlu_flan_cot_fewshot_abstract_algebra task: mmlu_flan_cot_fewshot_abstract_algebra
...@@ -70,6 +70,6 @@ fewshot_config: ...@@ -70,6 +70,6 @@ fewshot_config:
\ origin of the hyoid bone are the second and the third pharyngeal arches\u2014\ \ origin of the hyoid bone are the second and the third pharyngeal arches\u2014\
this information is covered in the last option (D). Therefore, we conclude that\ this information is covered in the last option (D). Therefore, we conclude that\
\ (D) must be the correct answer. The answer is (D).\n\n" \ (D) must be the correct answer. The answer is (D).\n\n"
group: mmlu_flan_cot_fewshot_stem tag: mmlu_flan_cot_fewshot_stem
include: _mmlu_flan_cot_fewshot_template_yaml include: _mmlu_flan_cot_fewshot_template_yaml
task: mmlu_flan_cot_fewshot_anatomy task: mmlu_flan_cot_fewshot_anatomy
...@@ -65,6 +65,6 @@ fewshot_config: ...@@ -65,6 +65,6 @@ fewshot_config:
because it explains that the surface is red due to the rusted materials on the because it explains that the surface is red due to the rusted materials on the
surface and the red color comes from the rust. So the correct option is (A). surface and the red color comes from the rust. So the correct option is (A).
The answer is (A).' The answer is (A).'
group: mmlu_flan_cot_fewshot_stem tag: mmlu_flan_cot_fewshot_stem
include: _mmlu_flan_cot_fewshot_template_yaml include: _mmlu_flan_cot_fewshot_template_yaml
task: mmlu_flan_cot_fewshot_astronomy task: mmlu_flan_cot_fewshot_astronomy
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment