Skip to content

GitLab

"src/vscode:/vscode.git/clone" did not exist on "658a32ab821d5cfbcb4037152768a2dff9a14a8d"
Commit 6c69e30a authored by FarzanehNakhaee's avatar FarzanehNakhaee
Browse files

Add qa4mre

parent cc3bdad3
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
lm_eval/tasks/qa4mre/qa4mre_2011.yaml
View file @ 6c69e30a
...@@ -9,7 +9,7 @@ template_aliases: "{% set answer_choices = answer_options['answer_str'] %}" ...@@ -9,7 +9,7 @@ template_aliases: "{% set answer_choices = answer_options['answer_str'] %}"
doc_to_text: "{{document_str.strip()}}\nQuestion:{{question_str}}\nAnswer:" doc_to_text: "{{document_str.strip()}}\nQuestion:{{question_str}}\nAnswer:"
doc_to_target: !function preprocess_qa4mre.qa4mre_process doc_to_target: !function preprocess_qa4mre.qa4mre_process
should_decontaminate: true should_decontaminate: true
doc_to_decontamination_query: "{{document_str|map('trim')}} + ' ' + {{question_str}}" doc_to_decontamination_query: "{{document_str.strip()}} + ' ' + {{question_str}}"
metric_list: metric_list:
- metric: acc - metric: acc
aggregation: mean aggregation: mean
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment