Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
8806944e
Commit
8806944e
authored
Oct 03, 2023
by
haileyschoelkopf
Browse files
push updated bigbench task yamls
parent
1a77b4d5
Changes
334
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
20 additions
and
20 deletions
+20
-20
lm_eval/tasks/bigbench/multiple_choice/geometric_shapes.yaml
lm_eval/tasks/bigbench/multiple_choice/geometric_shapes.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/goal_step_wikihow.yaml
...val/tasks/bigbench/multiple_choice/goal_step_wikihow.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/gre_reading_comprehension.yaml
...s/bigbench/multiple_choice/gre_reading_comprehension.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/hhh_alignment.yaml
lm_eval/tasks/bigbench/multiple_choice/hhh_alignment.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/hindi_question_answering.yaml
...ks/bigbench/multiple_choice/hindi_question_answering.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/hindu_knowledge.yaml
lm_eval/tasks/bigbench/multiple_choice/hindu_knowledge.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/hinglish_toxicity.yaml
...val/tasks/bigbench/multiple_choice/hinglish_toxicity.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/human_organs_senses.yaml
...l/tasks/bigbench/multiple_choice/human_organs_senses.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/hyperbaton.yaml
lm_eval/tasks/bigbench/multiple_choice/hyperbaton.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/identify_math_theorems.yaml
...asks/bigbench/multiple_choice/identify_math_theorems.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/identify_odd_metaphor.yaml
...tasks/bigbench/multiple_choice/identify_odd_metaphor.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/implicatures.yaml
lm_eval/tasks/bigbench/multiple_choice/implicatures.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/implicit_relations.yaml
...al/tasks/bigbench/multiple_choice/implicit_relations.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/intent_recognition.yaml
...al/tasks/bigbench/multiple_choice/intent_recognition.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_nli.yaml
.../multiple_choice/international_phonetic_alphabet_nli.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_transliterate.yaml
...choice/international_phonetic_alphabet_transliterate.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/intersect_geometry.yaml
...al/tasks/bigbench/multiple_choice/intersect_geometry.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/irony_identification.yaml
.../tasks/bigbench/multiple_choice/irony_identification.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/kanji_ascii.yaml
lm_eval/tasks/bigbench/multiple_choice/kanji_ascii.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/kannada.yaml
lm_eval/tasks/bigbench/multiple_choice/kannada.yaml
+1
-1
No files found.
lm_eval/tasks/bigbench/multiple_choice/geometric_shapes.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
geometric_shapes
dataset_name
:
geometric_shapes
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_geometric_shapes_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/goal_step_wikihow.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
goal_step_wikihow
dataset_name
:
goal_step_wikihow
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_goal_step_wikihow_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/gre_reading_comprehension.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
gre_reading_comprehension
dataset_name
:
gre_reading_comprehension
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_gre_reading_comprehension_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/hhh_alignment.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
hhh_alignment
dataset_name
:
hhh_alignment
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_hhh_alignment_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/hindi_question_answering.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
hindi_question_answering
dataset_name
:
hindi_question_answering
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_hindi_question_answering_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/hindu_knowledge.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
hindu_knowledge
dataset_name
:
hindu_knowledge
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_hindu_knowledge_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/hinglish_toxicity.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
hinglish_toxicity
dataset_name
:
hinglish_toxicity
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_hinglish_toxicity_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/human_organs_senses.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
human_organs_senses
dataset_name
:
human_organs_senses
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_human_organs_senses_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/hyperbaton.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
hyperbaton
dataset_name
:
hyperbaton
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_hyperbaton_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/identify_math_theorems.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
identify_math_theorems
dataset_name
:
identify_math_theorems
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_identify_math_theorems_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/identify_odd_metaphor.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
identify_odd_metaphor
dataset_name
:
identify_odd_metaphor
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_identify_odd_metaphor_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/implicatures.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
implicatures
dataset_name
:
implicatures
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_implicatures_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/implicit_relations.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
implicit_relations
dataset_name
:
implicit_relations
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_implicit_relations_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/intent_recognition.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
intent_recognition
dataset_name
:
intent_recognition
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_intent_recognition_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_nli.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
international_phonetic_alphabet_nli
dataset_name
:
international_phonetic_alphabet_nli
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_international_phonetic_alphabet_nli_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/international_phonetic_alphabet_transliterate.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
international_phonetic_alphabet_transliterate
dataset_name
:
international_phonetic_alphabet_transliterate
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_international_phonetic_alphabet_transliterate_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/intersect_geometry.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
intersect_geometry
dataset_name
:
intersect_geometry
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_intersect_geometry_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/irony_identification.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
irony_identification
dataset_name
:
irony_identification
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_irony_identification_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/kanji_ascii.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
kanji_ascii
dataset_name
:
kanji_ascii
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_kanji_ascii_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/kannada.yaml
View file @
8806944e
# Generated by utils.py
dataset_name
:
kannada
dataset_name
:
kannada
_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_kannada_multiple_choice
Prev
1
…
8
9
10
11
12
13
14
15
16
17
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment