Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
8806944e
Commit
8806944e
authored
Oct 03, 2023
by
haileyschoelkopf
Browse files
push updated bigbench task yamls
parent
1a77b4d5
Changes
334
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
20 additions
and
20 deletions
+20
-20
lm_eval/tasks/bigbench/multiple_choice/simp_turing_concept.yaml
...l/tasks/bigbench/multiple_choice/simp_turing_concept.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json.yaml
...asks/bigbench/multiple_choice/simple_arithmetic_json.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json_multiple_choice.yaml
...ltiple_choice/simple_arithmetic_json_multiple_choice.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json_subtasks.yaml
...ench/multiple_choice/simple_arithmetic_json_subtasks.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_multiple_targets_json.yaml
...tiple_choice/simple_arithmetic_multiple_targets_json.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/simple_ethical_questions.yaml
...ks/bigbench/multiple_choice/simple_ethical_questions.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/simple_text_editing.yaml
...l/tasks/bigbench/multiple_choice/simple_text_editing.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/snarks.yaml
lm_eval/tasks/bigbench/multiple_choice/snarks.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/social_iqa.yaml
lm_eval/tasks/bigbench/multiple_choice/social_iqa.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/social_support.yaml
lm_eval/tasks/bigbench/multiple_choice/social_support.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/sports_understanding.yaml
.../tasks/bigbench/multiple_choice/sports_understanding.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/strange_stories.yaml
lm_eval/tasks/bigbench/multiple_choice/strange_stories.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/strategyqa.yaml
lm_eval/tasks/bigbench/multiple_choice/strategyqa.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/sufficient_information.yaml
...asks/bigbench/multiple_choice/sufficient_information.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/suicide_risk.yaml
lm_eval/tasks/bigbench/multiple_choice/suicide_risk.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/swahili_english_proverbs.yaml
...ks/bigbench/multiple_choice/swahili_english_proverbs.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/swedish_to_german_proverbs.yaml
.../bigbench/multiple_choice/swedish_to_german_proverbs.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/symbol_interpretation.yaml
...tasks/bigbench/multiple_choice/symbol_interpretation.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/temporal_sequences.yaml
...al/tasks/bigbench/multiple_choice/temporal_sequences.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/tense.yaml
lm_eval/tasks/bigbench/multiple_choice/tense.yaml
+1
-1
No files found.
lm_eval/tasks/bigbench/multiple_choice/simp_turing_concept.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
simp_turing_concept
dataset_name
:
simp_turing_concept
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_simp_turing_concept_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
simple_arithmetic_json
dataset_name
:
simple_arithmetic_json
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_simple_arithmetic_json_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json_multiple_choice.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
simple_arithmetic_json_multiple_choice
dataset_name
:
simple_arithmetic_json_multiple_choice
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_simple_arithmetic_json_multiple_choice_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json_subtasks.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
simple_arithmetic_json_subtasks
dataset_name
:
simple_arithmetic_json_subtasks
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_simple_arithmetic_json_subtasks_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_multiple_targets_json.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
simple_arithmetic_multiple_targets_json
dataset_name
:
simple_arithmetic_multiple_targets_json
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_simple_arithmetic_multiple_targets_json_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/simple_ethical_questions.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
simple_ethical_questions
dataset_name
:
simple_ethical_questions
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_simple_ethical_questions_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/simple_text_editing.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
simple_text_editing
dataset_name
:
simple_text_editing
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_simple_text_editing_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/snarks.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
snarks
dataset_name
:
snarks
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_snarks_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/social_iqa.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
social_iqa
dataset_name
:
social_iqa
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_social_iqa_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/social_support.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
social_support
dataset_name
:
social_support
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_social_support_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/sports_understanding.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
sports_understanding
dataset_name
:
sports_understanding
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_sports_understanding_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/strange_stories.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
strange_stories
dataset_name
:
strange_stories
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_strange_stories_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/strategyqa.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
strategyqa
dataset_name
:
strategyqa
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_strategyqa_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/sufficient_information.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
sufficient_information
dataset_name
:
sufficient_information
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_sufficient_information_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/suicide_risk.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
suicide_risk
dataset_name
:
suicide_risk
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_suicide_risk_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/swahili_english_proverbs.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
swahili_english_proverbs
dataset_name
:
swahili_english_proverbs
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_swahili_english_proverbs_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/swedish_to_german_proverbs.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
swedish_to_german_proverbs
dataset_name
:
swedish_to_german_proverbs
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_swedish_to_german_proverbs_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/symbol_interpretation.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
symbol_interpretation
dataset_name
:
symbol_interpretation
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_symbol_interpretation_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/temporal_sequences.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
temporal_sequences
dataset_name
:
temporal_sequences
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_temporal_sequences_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/tense.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
tense
dataset_name
:
tense
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_tense_multiple_choice
Prev
1
…
12
13
14
15
16
17
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment