Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
8806944e
Commit
8806944e
authored
Oct 03, 2023
by
haileyschoelkopf
Browse files
push updated bigbench task yamls
parent
1a77b4d5
Changes
334
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
20 additions
and
20 deletions
+20
-20
lm_eval/tasks/bigbench/multiple_choice/modified_arithmetic.yaml
...l/tasks/bigbench/multiple_choice/modified_arithmetic.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/moral_permissibility.yaml
.../tasks/bigbench/multiple_choice/moral_permissibility.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/movie_dialog_same_or_different.yaml
...bench/multiple_choice/movie_dialog_same_or_different.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/movie_recommendation.yaml
.../tasks/bigbench/multiple_choice/movie_recommendation.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/mult_data_wrangling.yaml
...l/tasks/bigbench/multiple_choice/mult_data_wrangling.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/multiemo.yaml
lm_eval/tasks/bigbench/multiple_choice/multiemo.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/natural_instructions.yaml
.../tasks/bigbench/multiple_choice/natural_instructions.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/navigate.yaml
lm_eval/tasks/bigbench/multiple_choice/navigate.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml
...asks/bigbench/multiple_choice/nonsense_words_grammar.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml
lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/object_counting.yaml
lm_eval/tasks/bigbench/multiple_choice/object_counting.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml
lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/operators.yaml
lm_eval/tasks/bigbench/multiple_choice/operators.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/paragraph_segmentation.yaml
...asks/bigbench/multiple_choice/paragraph_segmentation.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml
lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/parsinlu_reading_comprehension.yaml
...bench/multiple_choice/parsinlu_reading_comprehension.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml
...l/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml
...val/tasks/bigbench/multiple_choice/periodic_elements.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml
lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml
...al/tasks/bigbench/multiple_choice/phrase_relatedness.yaml
+1
-1
No files found.
lm_eval/tasks/bigbench/multiple_choice/modified_arithmetic.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
modified_arithmetic
dataset_name
:
modified_arithmetic
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_modified_arithmetic_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/moral_permissibility.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
moral_permissibility
dataset_name
:
moral_permissibility
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_moral_permissibility_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/movie_dialog_same_or_different.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
movie_dialog_same_or_different
dataset_name
:
movie_dialog_same_or_different
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_movie_dialog_same_or_different_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/movie_recommendation.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
movie_recommendation
dataset_name
:
movie_recommendation
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_movie_recommendation_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/mult_data_wrangling.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
mult_data_wrangling
dataset_name
:
mult_data_wrangling
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_mult_data_wrangling_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/multiemo.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
multiemo
dataset_name
:
multiemo
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_multiemo_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/natural_instructions.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
natural_instructions
dataset_name
:
natural_instructions
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_natural_instructions_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/navigate.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
navigate
dataset_name
:
navigate
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_navigate_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
nonsense_words_grammar
dataset_name
:
nonsense_words_grammar
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_nonsense_words_grammar_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
novel_concepts
dataset_name
:
novel_concepts
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_novel_concepts_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/object_counting.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
object_counting
dataset_name
:
object_counting
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_object_counting_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
odd_one_out
dataset_name
:
odd_one_out
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_odd_one_out_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/operators.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
operators
dataset_name
:
operators
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_operators_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/paragraph_segmentation.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
paragraph_segmentation
dataset_name
:
paragraph_segmentation
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_paragraph_segmentation_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
parsinlu_qa
dataset_name
:
parsinlu_qa
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_parsinlu_qa_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/parsinlu_reading_comprehension.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
parsinlu_reading_comprehension
dataset_name
:
parsinlu_reading_comprehension
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_parsinlu_reading_comprehension_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
penguins_in_a_table
dataset_name
:
penguins_in_a_table
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_penguins_in_a_table_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
periodic_elements
dataset_name
:
periodic_elements
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_periodic_elements_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
persian_idioms
dataset_name
:
persian_idioms
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_persian_idioms_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
phrase_relatedness
dataset_name
:
phrase_relatedness
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_phrase_relatedness_multiple_choice
Prev
1
…
10
11
12
13
14
15
16
17
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment