Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
8806944e
Commit
8806944e
authored
Oct 03, 2023
by
haileyschoelkopf
Browse files
push updated bigbench task yamls
parent
1a77b4d5
Changes
334
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
20 additions
and
20 deletions
+20
-20
lm_eval/tasks/bigbench/multiple_choice/checkmate_in_one.yaml
lm_eval/tasks/bigbench/multiple_choice/checkmate_in_one.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/chess_state_tracking.yaml
.../tasks/bigbench/multiple_choice/chess_state_tracking.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/chinese_remainder_theorem.yaml
...s/bigbench/multiple_choice/chinese_remainder_theorem.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/cifar10_classification.yaml
...asks/bigbench/multiple_choice/cifar10_classification.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/code_line_description.yaml
...tasks/bigbench/multiple_choice/code_line_description.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/codenames.yaml
lm_eval/tasks/bigbench/multiple_choice/codenames.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/color.yaml
lm_eval/tasks/bigbench/multiple_choice/color.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/common_morpheme.yaml
lm_eval/tasks/bigbench/multiple_choice/common_morpheme.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/conceptual_combinations.yaml
...sks/bigbench/multiple_choice/conceptual_combinations.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/conlang_translation.yaml
...l/tasks/bigbench/multiple_choice/conlang_translation.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/contextual_parametric_knowledge_conflicts.yaml
...ple_choice/contextual_parametric_knowledge_conflicts.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/crash_blossom.yaml
lm_eval/tasks/bigbench/multiple_choice/crash_blossom.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/crass_ai.yaml
lm_eval/tasks/bigbench/multiple_choice/crass_ai.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/cryobiology_spanish.yaml
...l/tasks/bigbench/multiple_choice/cryobiology_spanish.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/cryptonite.yaml
lm_eval/tasks/bigbench/multiple_choice/cryptonite.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/cs_algorithms.yaml
lm_eval/tasks/bigbench/multiple_choice/cs_algorithms.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/dark_humor_detection.yaml
.../tasks/bigbench/multiple_choice/dark_humor_detection.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/date_understanding.yaml
...al/tasks/bigbench/multiple_choice/date_understanding.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/disambiguation_qa.yaml
...val/tasks/bigbench/multiple_choice/disambiguation_qa.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/discourse_marker_prediction.yaml
...bigbench/multiple_choice/discourse_marker_prediction.yaml
+1
-1
No files found.
lm_eval/tasks/bigbench/multiple_choice/checkmate_in_one.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
checkmate_in_one
dataset_name
:
checkmate_in_one
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_checkmate_in_one_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/chess_state_tracking.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
chess_state_tracking
dataset_name
:
chess_state_tracking
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_chess_state_tracking_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/chinese_remainder_theorem.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
chinese_remainder_theorem
dataset_name
:
chinese_remainder_theorem
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_chinese_remainder_theorem_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/cifar10_classification.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
cifar10_classification
dataset_name
:
cifar10_classification
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_cifar10_classification_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/code_line_description.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
code_line_description
dataset_name
:
code_line_description
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_code_line_description_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/codenames.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
codenames
dataset_name
:
codenames
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_codenames_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/color.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
color
dataset_name
:
color
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_color_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/common_morpheme.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
common_morpheme
dataset_name
:
common_morpheme
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_common_morpheme_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/conceptual_combinations.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
conceptual_combinations
dataset_name
:
conceptual_combinations
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_conceptual_combinations_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/conlang_translation.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
conlang_translation
dataset_name
:
conlang_translation
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_conlang_translation_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/contextual_parametric_knowledge_conflicts.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
contextual_parametric_knowledge_conflicts
dataset_name
:
contextual_parametric_knowledge_conflicts
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_contextual_parametric_knowledge_conflicts_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/crash_blossom.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
crash_blossom
dataset_name
:
crash_blossom
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_crash_blossom_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/crass_ai.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
crass_ai
dataset_name
:
crass_ai
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_crass_ai_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/cryobiology_spanish.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
cryobiology_spanish
dataset_name
:
cryobiology_spanish
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_cryobiology_spanish_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/cryptonite.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
cryptonite
dataset_name
:
cryptonite
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_cryptonite_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/cs_algorithms.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
cs_algorithms
dataset_name
:
cs_algorithms
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_cs_algorithms_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/dark_humor_detection.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
dark_humor_detection
dataset_name
:
dark_humor_detection
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_dark_humor_detection_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/date_understanding.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
date_understanding
dataset_name
:
date_understanding
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_date_understanding_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/disambiguation_qa.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
disambiguation_qa
dataset_name
:
disambiguation_qa
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_disambiguation_qa_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/discourse_marker_prediction.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
discourse_marker_prediction
dataset_name
:
discourse_marker_prediction
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_discourse_marker_prediction_multiple_choice
Prev
1
…
6
7
8
9
10
11
12
13
14
…
17
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment