Commit 09459176 authored by FarzanehNakhaee's avatar FarzanehNakhaee
Browse files

Upload qa4mre_2012

parent 9907e0a7
group:
- multiple_choice
task: qa4mre_2012
dataset_path: qa4mre
dataset_name: 2012.main.EN
output_type: multiple_choice
test_split: train
template_aliases: "{% set answer_choices = answer_options['answer_str'] %}"
doc_to_text: "{{document_str.strip()}}\nQuestion: {{question_str}}\nAnswer:"
doc_to_target: !function preprocess_qa4mre.qa4mre_process
should_decontaminate: true
doc_to_decontamination_query: "{{document_str.strip()}} + ' ' + {{question_str}}"
metric_list:
- metric: acc
aggregation: mean
higher_is_better: true
- metric: acc_norm
aggregation: mean
higher_is_better: true
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment