Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
8806944e
Commit
8806944e
authored
Oct 03, 2023
by
haileyschoelkopf
Browse files
push updated bigbench task yamls
parent
1a77b4d5
Changes
334
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
20 additions
and
20 deletions
+20
-20
lm_eval/tasks/bigbench/greedy_until/unnatural_in_context_learning.yaml
.../bigbench/greedy_until/unnatural_in_context_learning.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/vitaminc_fact_verification.yaml
...sks/bigbench/greedy_until/vitaminc_fact_verification.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/what_is_the_tao.yaml
lm_eval/tasks/bigbench/greedy_until/what_is_the_tao.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/which_wiki_edit.yaml
lm_eval/tasks/bigbench/greedy_until/which_wiki_edit.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/winowhy.yaml
lm_eval/tasks/bigbench/greedy_until/winowhy.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/word_sorting.yaml
lm_eval/tasks/bigbench/greedy_until/word_sorting.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/word_unscrambling.yaml
lm_eval/tasks/bigbench/greedy_until/word_unscrambling.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/abstract_narrative_understanding.yaml
...nch/multiple_choice/abstract_narrative_understanding.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/anachronisms.yaml
lm_eval/tasks/bigbench/multiple_choice/anachronisms.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/analogical_similarity.yaml
...tasks/bigbench/multiple_choice/analogical_similarity.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/analytic_entailment.yaml
...l/tasks/bigbench/multiple_choice/analytic_entailment.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/arithmetic.yaml
lm_eval/tasks/bigbench/multiple_choice/arithmetic.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/ascii_word_recognition.yaml
...asks/bigbench/multiple_choice/ascii_word_recognition.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/authorship_verification.yaml
...sks/bigbench/multiple_choice/authorship_verification.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/auto_categorization.yaml
...l/tasks/bigbench/multiple_choice/auto_categorization.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/auto_debugging.yaml
lm_eval/tasks/bigbench/multiple_choice/auto_debugging.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/bbq_lite_json.yaml
lm_eval/tasks/bigbench/multiple_choice/bbq_lite_json.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/bridging_anaphora_resolution_barqa.yaml
...h/multiple_choice/bridging_anaphora_resolution_barqa.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/causal_judgment.yaml
lm_eval/tasks/bigbench/multiple_choice/causal_judgment.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/cause_and_effect.yaml
lm_eval/tasks/bigbench/multiple_choice/cause_and_effect.yaml
+1
-1
No files found.
lm_eval/tasks/bigbench/greedy_until/unnatural_in_context_learning.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
unnatural_in_context_learning
dataset_name
:
unnatural_in_context_learning
_zero_shot
include
:
../greedy_until_template_yaml
task
:
bigbench_unnatural_in_context_learning_greedy_until
lm_eval/tasks/bigbench/greedy_until/vitaminc_fact_verification.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
vitaminc_fact_verification
dataset_name
:
vitaminc_fact_verification
_zero_shot
include
:
../greedy_until_template_yaml
task
:
bigbench_vitaminc_fact_verification_greedy_until
lm_eval/tasks/bigbench/greedy_until/what_is_the_tao.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
what_is_the_tao
dataset_name
:
what_is_the_tao
_zero_shot
include
:
../greedy_until_template_yaml
task
:
bigbench_what_is_the_tao_greedy_until
lm_eval/tasks/bigbench/greedy_until/which_wiki_edit.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
which_wiki_edit
dataset_name
:
which_wiki_edit
_zero_shot
include
:
../greedy_until_template_yaml
task
:
bigbench_which_wiki_edit_greedy_until
lm_eval/tasks/bigbench/greedy_until/winowhy.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
winowhy
dataset_name
:
winowhy
_zero_shot
include
:
../greedy_until_template_yaml
task
:
bigbench_winowhy_greedy_until
lm_eval/tasks/bigbench/greedy_until/word_sorting.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
word_sorting
dataset_name
:
word_sorting
_zero_shot
include
:
../greedy_until_template_yaml
task
:
bigbench_word_sorting_greedy_until
lm_eval/tasks/bigbench/greedy_until/word_unscrambling.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
word_unscrambling
dataset_name
:
word_unscrambling
_zero_shot
include
:
../greedy_until_template_yaml
task
:
bigbench_word_unscrambling_greedy_until
lm_eval/tasks/bigbench/multiple_choice/abstract_narrative_understanding.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
abstract_narrative_understanding
dataset_name
:
abstract_narrative_understanding
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_abstract_narrative_understanding_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/anachronisms.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
anachronisms
dataset_name
:
anachronisms
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_anachronisms_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/analogical_similarity.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
analogical_similarity
dataset_name
:
analogical_similarity
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_analogical_similarity_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/analytic_entailment.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
analytic_entailment
dataset_name
:
analytic_entailment
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_analytic_entailment_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/arithmetic.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
arithmetic
dataset_name
:
arithmetic
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_arithmetic_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/ascii_word_recognition.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
ascii_word_recognition
dataset_name
:
ascii_word_recognition
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_ascii_word_recognition_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/authorship_verification.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
authorship_verification
dataset_name
:
authorship_verification
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_authorship_verification_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/auto_categorization.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
auto_categorization
dataset_name
:
auto_categorization
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_auto_categorization_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/auto_debugging.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
auto_debugging
dataset_name
:
auto_debugging
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_auto_debugging_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/bbq_lite_json.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
bbq_lite_json
dataset_name
:
bbq_lite_json
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_bbq_lite_json_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/bridging_anaphora_resolution_barqa.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
bridging_anaphora_resolution_barqa
dataset_name
:
bridging_anaphora_resolution_barqa
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_bridging_anaphora_resolution_barqa_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/causal_judgment.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
causal_judgment
dataset_name
:
causal_judgment
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_causal_judgment_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/cause_and_effect.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
cause_and_effect
dataset_name
:
cause_and_effect
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_cause_and_effect_multiple_choice
Prev
1
…
5
6
7
8
9
10
11
12
13
…
17
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment