Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
de71ad92
Unverified
Commit
de71ad92
authored
Oct 17, 2023
by
Lintang Sutawika
Committed by
GitHub
Oct 17, 2023
Browse files
Merge branch 'big-refactor' into fix-unittests
parents
09d20bfa
73c80915
Changes
370
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
80 additions
and
0 deletions
+80
-0
lm_eval/tasks/bigbench/generate_until/general_knowledge.yaml
lm_eval/tasks/bigbench/generate_until/general_knowledge.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/geometric_shapes.yaml
lm_eval/tasks/bigbench/generate_until/geometric_shapes.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/goal_step_wikihow.yaml
lm_eval/tasks/bigbench/generate_until/goal_step_wikihow.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/gre_reading_comprehension.yaml
...ks/bigbench/generate_until/gre_reading_comprehension.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/hhh_alignment.yaml
lm_eval/tasks/bigbench/generate_until/hhh_alignment.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/hindi_question_answering.yaml
...sks/bigbench/generate_until/hindi_question_answering.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/hindu_knowledge.yaml
lm_eval/tasks/bigbench/generate_until/hindu_knowledge.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/hinglish_toxicity.yaml
lm_eval/tasks/bigbench/generate_until/hinglish_toxicity.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/human_organs_senses.yaml
...al/tasks/bigbench/generate_until/human_organs_senses.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/hyperbaton.yaml
lm_eval/tasks/bigbench/generate_until/hyperbaton.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/identify_math_theorems.yaml
...tasks/bigbench/generate_until/identify_math_theorems.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/identify_odd_metaphor.yaml
.../tasks/bigbench/generate_until/identify_odd_metaphor.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/implicatures.yaml
lm_eval/tasks/bigbench/generate_until/implicatures.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/implicit_relations.yaml
...val/tasks/bigbench/generate_until/implicit_relations.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/intent_recognition.yaml
...val/tasks/bigbench/generate_until/intent_recognition.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_nli.yaml
...h/generate_until/international_phonetic_alphabet_nli.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/international_phonetic_alphabet_transliterate.yaml
..._until/international_phonetic_alphabet_transliterate.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/intersect_geometry.yaml
...val/tasks/bigbench/generate_until/intersect_geometry.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/irony_identification.yaml
...l/tasks/bigbench/generate_until/irony_identification.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/kanji_ascii.yaml
lm_eval/tasks/bigbench/generate_until/kanji_ascii.yaml
+4
-0
No files found.
lm_eval/tasks/bigbench/g
reedy
_until/general_knowledge.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/general_knowledge.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
general_knowledge_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_general_knowledge_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_general_knowledge_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/geometric_shapes.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/geometric_shapes.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
geometric_shapes_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_geometric_shapes_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_geometric_shapes_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/goal_step_wikihow.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/goal_step_wikihow.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
goal_step_wikihow_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_goal_step_wikihow_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_goal_step_wikihow_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/gre_reading_comprehension.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/gre_reading_comprehension.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
gre_reading_comprehension_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_gre_reading_comprehension_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_gre_reading_comprehension_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/hhh_alignment.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/hhh_alignment.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
hhh_alignment_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_hhh_alignment_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_hhh_alignment_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/hindi_question_answering.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/hindi_question_answering.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
hindi_question_answering_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_hindi_question_answering_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_hindi_question_answering_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/hindu_knowledge.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/hindu_knowledge.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
hindu_knowledge_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_hindu_knowledge_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_hindu_knowledge_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/hinglish_toxicity.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/hinglish_toxicity.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
hinglish_toxicity_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_hinglish_toxicity_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_hinglish_toxicity_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/human_organs_senses.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/human_organs_senses.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
human_organs_senses_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_human_organs_senses_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_human_organs_senses_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/hyperbaton.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/hyperbaton.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
hyperbaton_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_hyperbaton_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_hyperbaton_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/identify_math_theorems.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/identify_math_theorems.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
identify_math_theorems_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_identify_math_theorems_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_identify_math_theorems_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/identify_odd_metaphor.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/identify_odd_metaphor.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
identify_odd_metaphor_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_identify_odd_metaphor_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_identify_odd_metaphor_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/implicatures.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/implicatures.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
implicatures_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_implicatures_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_implicatures_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/implicit_relations.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/implicit_relations.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
implicit_relations_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_implicit_relations_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_implicit_relations_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/intent_recognition.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/intent_recognition.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
intent_recognition_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_intent_recognition_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_intent_recognition_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/international_phonetic_alphabet_nli.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/international_phonetic_alphabet_nli.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
international_phonetic_alphabet_nli_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_international_phonetic_alphabet_nli_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_international_phonetic_alphabet_nli_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/international_phonetic_alphabet_transliterate.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/international_phonetic_alphabet_transliterate.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
international_phonetic_alphabet_transliterate_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_international_phonetic_alphabet_transliterate_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_international_phonetic_alphabet_transliterate_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/intersect_geometry.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/intersect_geometry.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
intersect_geometry_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_intersect_geometry_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_intersect_geometry_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/irony_identification.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/irony_identification.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
irony_identification_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_irony_identification_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_irony_identification_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/kanji_ascii.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/kanji_ascii.yaml
View file @
de71ad92
# Generated by utils.py
dataset_name
:
kanji_ascii_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_kanji_ascii_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_kanji_ascii_g
enerate
_until
Prev
1
…
7
8
9
10
11
12
13
14
15
…
19
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment