Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
605787a9
Commit
605787a9
authored
Sep 13, 2023
by
lintangsutawika
Browse files
add bigbench
parent
b8d1cef9
Changes
338
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
80 additions
and
0 deletions
+80
-0
lm_eval/tasks/bigbench/greedy_until/linguistics_puzzles.yaml
lm_eval/tasks/bigbench/greedy_until/linguistics_puzzles.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/list_functions.yaml
lm_eval/tasks/bigbench/greedy_until/list_functions.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/logic_grid_puzzle.yaml
lm_eval/tasks/bigbench/greedy_until/logic_grid_puzzle.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/logical_args.yaml
lm_eval/tasks/bigbench/greedy_until/logical_args.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/logical_deduction.yaml
lm_eval/tasks/bigbench/greedy_until/logical_deduction.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/logical_fallacy_detection.yaml
...asks/bigbench/greedy_until/logical_fallacy_detection.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/logical_sequence.yaml
lm_eval/tasks/bigbench/greedy_until/logical_sequence.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/mathematical_induction.yaml
...l/tasks/bigbench/greedy_until/mathematical_induction.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/matrixshapes.yaml
lm_eval/tasks/bigbench/greedy_until/matrixshapes.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/metaphor_boolean.yaml
lm_eval/tasks/bigbench/greedy_until/metaphor_boolean.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/metaphor_understanding.yaml
...l/tasks/bigbench/greedy_until/metaphor_understanding.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/minute_mysteries_qa.yaml
lm_eval/tasks/bigbench/greedy_until/minute_mysteries_qa.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/misconceptions.yaml
lm_eval/tasks/bigbench/greedy_until/misconceptions.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/misconceptions_russian.yaml
...l/tasks/bigbench/greedy_until/misconceptions_russian.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/mnist_ascii.yaml
lm_eval/tasks/bigbench/greedy_until/mnist_ascii.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/modified_arithmetic.yaml
lm_eval/tasks/bigbench/greedy_until/modified_arithmetic.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/moral_permissibility.yaml
...val/tasks/bigbench/greedy_until/moral_permissibility.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/movie_dialog_same_or_different.yaml
...bigbench/greedy_until/movie_dialog_same_or_different.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/movie_recommendation.yaml
...val/tasks/bigbench/greedy_until/movie_recommendation.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/mult_data_wrangling.yaml
lm_eval/tasks/bigbench/greedy_until/mult_data_wrangling.yaml
+4
-0
No files found.
lm_eval/tasks/bigbench/greedy_until/linguistics_puzzles.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
linguistics_puzzles
include
:
../greedy_until_template_yaml
task
:
bigbench_linguistics_puzzles_greedy_until
lm_eval/tasks/bigbench/greedy_until/list_functions.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
list_functions
include
:
../greedy_until_template_yaml
task
:
bigbench_list_functions_greedy_until
lm_eval/tasks/bigbench/greedy_until/logic_grid_puzzle.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
logic_grid_puzzle
include
:
../greedy_until_template_yaml
task
:
bigbench_logic_grid_puzzle_greedy_until
lm_eval/tasks/bigbench/greedy_until/logical_args.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
logical_args
include
:
../greedy_until_template_yaml
task
:
bigbench_logical_args_greedy_until
lm_eval/tasks/bigbench/greedy_until/logical_deduction.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
logical_deduction
include
:
../greedy_until_template_yaml
task
:
bigbench_logical_deduction_greedy_until
lm_eval/tasks/bigbench/greedy_until/logical_fallacy_detection.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
logical_fallacy_detection
include
:
../greedy_until_template_yaml
task
:
bigbench_logical_fallacy_detection_greedy_until
lm_eval/tasks/bigbench/greedy_until/logical_sequence.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
logical_sequence
include
:
../greedy_until_template_yaml
task
:
bigbench_logical_sequence_greedy_until
lm_eval/tasks/bigbench/greedy_until/mathematical_induction.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
mathematical_induction
include
:
../greedy_until_template_yaml
task
:
bigbench_mathematical_induction_greedy_until
lm_eval/tasks/bigbench/greedy_until/matrixshapes.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
matrixshapes
include
:
../greedy_until_template_yaml
task
:
bigbench_matrixshapes_greedy_until
lm_eval/tasks/bigbench/greedy_until/metaphor_boolean.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
metaphor_boolean
include
:
../greedy_until_template_yaml
task
:
bigbench_metaphor_boolean_greedy_until
lm_eval/tasks/bigbench/greedy_until/metaphor_understanding.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
metaphor_understanding
include
:
../greedy_until_template_yaml
task
:
bigbench_metaphor_understanding_greedy_until
lm_eval/tasks/bigbench/greedy_until/minute_mysteries_qa.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
minute_mysteries_qa
include
:
../greedy_until_template_yaml
task
:
bigbench_minute_mysteries_qa_greedy_until
lm_eval/tasks/bigbench/greedy_until/misconceptions.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
misconceptions
include
:
../greedy_until_template_yaml
task
:
bigbench_misconceptions_greedy_until
lm_eval/tasks/bigbench/greedy_until/misconceptions_russian.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
misconceptions_russian
include
:
../greedy_until_template_yaml
task
:
bigbench_misconceptions_russian_greedy_until
lm_eval/tasks/bigbench/greedy_until/mnist_ascii.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
mnist_ascii
include
:
../greedy_until_template_yaml
task
:
bigbench_mnist_ascii_greedy_until
lm_eval/tasks/bigbench/greedy_until/modified_arithmetic.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
modified_arithmetic
include
:
../greedy_until_template_yaml
task
:
bigbench_modified_arithmetic_greedy_until
lm_eval/tasks/bigbench/greedy_until/moral_permissibility.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
moral_permissibility
include
:
../greedy_until_template_yaml
task
:
bigbench_moral_permissibility_greedy_until
lm_eval/tasks/bigbench/greedy_until/movie_dialog_same_or_different.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
movie_dialog_same_or_different
include
:
../greedy_until_template_yaml
task
:
bigbench_movie_dialog_same_or_different_greedy_until
lm_eval/tasks/bigbench/greedy_until/movie_recommendation.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
movie_recommendation
include
:
../greedy_until_template_yaml
task
:
bigbench_movie_recommendation_greedy_until
lm_eval/tasks/bigbench/greedy_until/mult_data_wrangling.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
mult_data_wrangling
include
:
../greedy_until_template_yaml
task
:
bigbench_mult_data_wrangling_greedy_until
Prev
1
2
3
4
5
6
7
8
9
…
17
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment