Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
e408c2ce
Commit
e408c2ce
authored
Nov 27, 2023
by
haileyschoelkopf
Browse files
make 'bbh' a group name for flan cot 3-shot, add stopseqs
parent
b189066d
Changes
4
Hide whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
11 additions
and
1 deletion
+11
-1
lm_eval/tasks/bbh/flan_cot_fewshot/_flan_cot_fewshot_template_yaml
...asks/bbh/flan_cot_fewshot/_flan_cot_fewshot_template_yaml
+5
-1
lm_eval/tasks/bbh/flan_cot_zeroshot/_flan_cot_zeroshot_template_yaml
...ks/bbh/flan_cot_zeroshot/_flan_cot_zeroshot_template_yaml
+2
-0
lm_eval/tasks/bbh/flan_fewshot/_flan_fewshot_template_yaml
lm_eval/tasks/bbh/flan_fewshot/_flan_fewshot_template_yaml
+2
-0
lm_eval/tasks/bbh/flan_zeroshot/_flan_zeroshot_template_yaml
lm_eval/tasks/bbh/flan_zeroshot/_flan_zeroshot_template_yaml
+2
-0
No files found.
lm_eval/tasks/bbh/flan_cot_fewshot/_flan_cot_fewshot_template_yaml
View file @
e408c2ce
group: bbh_flan_cot_fewshot
group:
- bbh_flan_cot_fewshot
- bbh
dataset_path: lukaemon/bbh
output_type: generate_until
test_split: test
...
...
@@ -12,6 +14,8 @@ metric_list:
generation_kwargs:
until:
- "</s>"
- "Q"
- "\n\n"
do_sample: false
temperature: 0.0
filter_list:
...
...
lm_eval/tasks/bbh/flan_cot_zeroshot/_flan_cot_zeroshot_template_yaml
View file @
e408c2ce
...
...
@@ -12,6 +12,8 @@ metric_list:
generation_kwargs:
until:
- "</s>"
- "Q"
- "\n\n"
do_sample: false
temperature: 0.0
filter_list:
...
...
lm_eval/tasks/bbh/flan_fewshot/_flan_fewshot_template_yaml
View file @
e408c2ce
...
...
@@ -12,5 +12,7 @@ metric_list:
generation_kwargs:
until:
- "</s>"
- "Q"
- "\n\n"
do_sample: false
temperature: 0.0
lm_eval/tasks/bbh/flan_zeroshot/_flan_zeroshot_template_yaml
View file @
e408c2ce
...
...
@@ -12,5 +12,7 @@ metric_list:
generation_kwargs:
until:
- "</s>"
- "Q:"
- "\n\n"
do_sample: false
temperature: 0.0
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment