Skip to content

GitLab

Commit e408c2ce authored by haileyschoelkopf's avatar haileyschoelkopf
Browse files

make 'bbh' a group name for flan cot 3-shot, add stopseqs

parent b189066d
Hide whitespace changes
Inline Side-by-side
Showing with 11 additions and 1 deletion
lm_eval/tasks/bbh/flan_cot_fewshot/_flan_cot_fewshot_template_yaml
View file @ e408c2ce
group: bbh_flan_cot_fewshot group:
- bbh_flan_cot_fewshot
- bbh
dataset_path: lukaemon/bbh dataset_path: lukaemon/bbh
output_type: generate_until output_type: generate_until
test_split: test test_split: test
...@@ -12,6 +14,8 @@ metric_list: ...@@ -12,6 +14,8 @@ metric_list:
generation_kwargs: generation_kwargs:
until: until:
- "</s>" - "</s>"
- "Q"
- "\n\n"
do_sample: false do_sample: false
temperature: 0.0 temperature: 0.0
filter_list: filter_list:
......
lm_eval/tasks/bbh/flan_cot_zeroshot/_flan_cot_zeroshot_template_yaml
View file @ e408c2ce
...@@ -12,6 +12,8 @@ metric_list: ...@@ -12,6 +12,8 @@ metric_list:
generation_kwargs: generation_kwargs:
until: until:
- "</s>" - "</s>"
- "Q"
- "\n\n"
do_sample: false do_sample: false
temperature: 0.0 temperature: 0.0
filter_list: filter_list:
......
lm_eval/tasks/bbh/flan_fewshot/_flan_fewshot_template_yaml
View file @ e408c2ce
...@@ -12,5 +12,7 @@ metric_list: ...@@ -12,5 +12,7 @@ metric_list:
generation_kwargs: generation_kwargs:
until: until:
- "</s>" - "</s>"
- "Q"
- "\n\n"
do_sample: false do_sample: false
temperature: 0.0 temperature: 0.0
lm_eval/tasks/bbh/flan_zeroshot/_flan_zeroshot_template_yaml
View file @ e408c2ce
...@@ -12,5 +12,7 @@ metric_list: ...@@ -12,5 +12,7 @@ metric_list:
generation_kwargs: generation_kwargs:
until: until:
- "</s>" - "</s>"
- "Q:"
- "\n\n"
do_sample: false do_sample: false
temperature: 0.0 temperature: 0.0
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment