Commit a4b6f914 authored by Benjamin Fattori's avatar Benjamin Fattori
Browse files

SWAG completed

parent 886190e9
group:
- multiple_choice
task: swag
dataset_path: swag
dataset_name: regular
output_type: multiple_choice
training_split: train
validation_split: validation
test_split: null
template_aliases: "{% set answer_choices = [ending0, ending1, ending2, ending3] %}{% set gold = label %}"
doc_to_text: "{{startphrase}}"
doc_to_target: "{{answer_choices[gold]}}"
gold_alias: "{{gold}}"
metric_list:
- metric: acc
aggregation: mean
higher_is_better: true
- metric: acc_norm
aggregation: mean
higher_is_better: true
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment