Skip to content

GitLab

"vscode:/vscode.git/clone" did not exist on "27ecc17a1ea93b5b6b68145df06094de0aa53356"
Commit 5dd41ebe authored by lintangsutawika's avatar lintangsutawika
Browse files

fixes and added decontaminate args

parent f2b52d26
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 1 deletion
lm_eval/tasks/piqa/piqa.yaml
View file @ 5dd41ebe
...@@ -7,9 +7,11 @@ output_type: multiple_choice ...@@ -7,9 +7,11 @@ output_type: multiple_choice
training_split: train training_split: train
validation_split: validation validation_split: validation
test_split: null test_split: null
doc_to_text: "Question: {{question}}\nAnswer:" doc_to_text: "Question: {{goal}}\nAnswer:"
doc_to_target: label doc_to_target: label
doc_to_choice: "{{[sol1, sol2]}}" doc_to_choice: "{{[sol1, sol2]}}"
should_decontaminate: true
doc_to_decontamination_query: goal
metric_list: metric_list:
- metric: acc - metric: acc
aggregation: mean aggregation: mean
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment