Skip to content

GitLab

Unverified Commit 3d2ee4d4 authored by Hailey Schoelkopf's avatar Hailey Schoelkopf Committed by GitHub
Browse files

Update _default_template_yaml

parent 6ae9e9e8
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
lm_eval/tasks/mmlu/default/_default_template_yaml
View file @ 3d2ee4d4
...@@ -2,6 +2,8 @@ group: mmlu ...@@ -2,6 +2,8 @@ group: mmlu
dataset_path: cais/mmlu dataset_path: cais/mmlu
test_split: test test_split: test
fewshot_split: dev fewshot_split: dev
fewshot_config:
sampler: first_n
output_type: multiple_choice output_type: multiple_choice
doc_to_text: "{{question.strip()}}\nA. {{choices[0]}}\nB. {{choices[1]}}\nC. {{choices[2]}}\nD. {{choices[3]}}\nAnswer:" doc_to_text: "{{question.strip()}}\nA. {{choices[0]}}\nB. {{choices[1]}}\nC. {{choices[2]}}\nD. {{choices[3]}}\nAnswer:"
doc_to_choice: ["A", "B", "C", "D"] doc_to_choice: ["A", "B", "C", "D"]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment