Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
8806944e
Commit
8806944e
authored
Oct 03, 2023
by
haileyschoelkopf
Browse files
push updated bigbench task yamls
parent
1a77b4d5
Changes
334
Hide whitespace changes
Inline
Side-by-side
Showing
14 changed files
with
14 additions
and
14 deletions
+14
-14
lm_eval/tasks/bigbench/multiple_choice/timedial.yaml
lm_eval/tasks/bigbench/multiple_choice/timedial.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/topical_chat.yaml
lm_eval/tasks/bigbench/multiple_choice/topical_chat.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/tracking_shuffled_objects.yaml
...s/bigbench/multiple_choice/tracking_shuffled_objects.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/understanding_fables.yaml
.../tasks/bigbench/multiple_choice/understanding_fables.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml
lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml
lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/unit_interpretation.yaml
...l/tasks/bigbench/multiple_choice/unit_interpretation.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/unnatural_in_context_learning.yaml
...gbench/multiple_choice/unnatural_in_context_learning.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/vitaminc_fact_verification.yaml
.../bigbench/multiple_choice/vitaminc_fact_verification.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/what_is_the_tao.yaml
lm_eval/tasks/bigbench/multiple_choice/what_is_the_tao.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/which_wiki_edit.yaml
lm_eval/tasks/bigbench/multiple_choice/which_wiki_edit.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/winowhy.yaml
lm_eval/tasks/bigbench/multiple_choice/winowhy.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/word_sorting.yaml
lm_eval/tasks/bigbench/multiple_choice/word_sorting.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/word_unscrambling.yaml
...val/tasks/bigbench/multiple_choice/word_unscrambling.yaml
+1
-1
No files found.
lm_eval/tasks/bigbench/multiple_choice/timedial.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
timedial
dataset_name
:
timedial
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_timedial_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/topical_chat.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
topical_chat
dataset_name
:
topical_chat
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_topical_chat_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/tracking_shuffled_objects.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
tracking_shuffled_objects
dataset_name
:
tracking_shuffled_objects
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_tracking_shuffled_objects_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/understanding_fables.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
understanding_fables
dataset_name
:
understanding_fables
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_understanding_fables_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
undo_permutation
dataset_name
:
undo_permutation
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_undo_permutation_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
unit_conversion
dataset_name
:
unit_conversion
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_unit_conversion_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/unit_interpretation.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
unit_interpretation
dataset_name
:
unit_interpretation
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_unit_interpretation_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/unnatural_in_context_learning.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
unnatural_in_context_learning
dataset_name
:
unnatural_in_context_learning
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_unnatural_in_context_learning_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/vitaminc_fact_verification.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
vitaminc_fact_verification
dataset_name
:
vitaminc_fact_verification
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_vitaminc_fact_verification_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/what_is_the_tao.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
what_is_the_tao
dataset_name
:
what_is_the_tao
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_what_is_the_tao_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/which_wiki_edit.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
which_wiki_edit
dataset_name
:
which_wiki_edit
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_which_wiki_edit_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/winowhy.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
winowhy
dataset_name
:
winowhy
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_winowhy_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/word_sorting.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
word_sorting
dataset_name
:
word_sorting
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_word_sorting_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/word_unscrambling.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
word_unscrambling
dataset_name
:
word_unscrambling
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_word_unscrambling_multiple_choice
Prev
1
…
13
14
15
16
17
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment