Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
3e1301bb
"...lm-evaluation-harness.git" did not exist on "ddb7c0f3c9590b1c61381cc0d1846f08aa0b0f99"
Commit
3e1301bb
authored
Jun 04, 2024
by
lintangsutawika
Browse files
resolved merge conflict from latest version
parents
fd9cd80f
070d31df
Changes
539
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
13 additions
and
41 deletions
+13
-41
lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml
...asks/bigbench/multiple_choice/nonsense_words_grammar.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml
lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/object_counting.yaml
lm_eval/tasks/bigbench/multiple_choice/object_counting.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml
lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/operators.yaml
lm_eval/tasks/bigbench/multiple_choice/operators.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/paragraph_segmentation.yaml
...asks/bigbench/multiple_choice/paragraph_segmentation.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml
lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/parsinlu_reading_comprehension.yaml
...bench/multiple_choice/parsinlu_reading_comprehension.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml
...l/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml
...val/tasks/bigbench/multiple_choice/periodic_elements.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml
lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml
...al/tasks/bigbench/multiple_choice/phrase_relatedness.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/physical_intuition.yaml
...al/tasks/bigbench/multiple_choice/physical_intuition.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/physics.yaml
lm_eval/tasks/bigbench/multiple_choice/physics.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/physics_questions.yaml
...val/tasks/bigbench/multiple_choice/physics_questions.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/play_dialog_same_or_different.yaml
...gbench/multiple_choice/play_dialog_same_or_different.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/polish_sequence_labeling.yaml
...ks/bigbench/multiple_choice/polish_sequence_labeling.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/presuppositions_as_nli.yaml
...asks/bigbench/multiple_choice/presuppositions_as_nli.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/qa_wikidata.yaml
lm_eval/tasks/bigbench/multiple_choice/qa_wikidata.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/question_selection.yaml
...al/tasks/bigbench/multiple_choice/question_selection.yaml
+1
-1
No files found.
lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
nonsense_words_grammar_zero_shot
dataset_name
:
nonsense_words_grammar_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_nonsense_words_grammar_multiple_choice
task
:
bigbench_nonsense_words_grammar_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
novel_concepts_zero_shot
dataset_name
:
novel_concepts_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_novel_concepts_multiple_choice
task
:
bigbench_novel_concepts_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/object_counting.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
object_counting_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_object_counting_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
odd_one_out_zero_shot
dataset_name
:
odd_one_out_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_odd_one_out_multiple_choice
task
:
bigbench_odd_one_out_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/operators.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
operators_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_operators_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/paragraph_segmentation.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
paragraph_segmentation_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_paragraph_segmentation_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
parsinlu_qa_zero_shot
dataset_name
:
parsinlu_qa_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_parsinlu_qa_multiple_choice
task
:
bigbench_parsinlu_qa_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/parsinlu_reading_comprehension.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
parsinlu_reading_comprehension_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_parsinlu_reading_comprehension_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
penguins_in_a_table_zero_shot
dataset_name
:
penguins_in_a_table_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_penguins_in_a_table_multiple_choice
task
:
bigbench_penguins_in_a_table_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
periodic_elements_zero_shot
dataset_name
:
periodic_elements_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_periodic_elements_multiple_choice
task
:
bigbench_periodic_elements_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
persian_idioms_zero_shot
dataset_name
:
persian_idioms_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_persian_idioms_multiple_choice
task
:
bigbench_persian_idioms_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
phrase_relatedness_zero_shot
dataset_name
:
phrase_relatedness_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_phrase_relatedness_multiple_choice
task
:
bigbench_phrase_relatedness_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/physical_intuition.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
physical_intuition_zero_shot
dataset_name
:
physical_intuition_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_physical_intuition_multiple_choice
task
:
bigbench_physical_intuition_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/physics.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
physics_zero_shot
dataset_name
:
physics_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_physics_multiple_choice
task
:
bigbench_physics_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/physics_questions.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
physics_questions_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_physics_questions_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/play_dialog_same_or_different.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
play_dialog_same_or_different_zero_shot
dataset_name
:
play_dialog_same_or_different_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_play_dialog_same_or_different_multiple_choice
task
:
bigbench_play_dialog_same_or_different_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/polish_sequence_labeling.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
polish_sequence_labeling_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_polish_sequence_labeling_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/presuppositions_as_nli.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
presuppositions_as_nli_zero_shot
dataset_name
:
presuppositions_as_nli_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_presuppositions_as_nli_multiple_choice
task
:
bigbench_presuppositions_as_nli_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/qa_wikidata.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
qa_wikidata_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_qa_wikidata_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/question_selection.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
question_selection_zero_shot
dataset_name
:
question_selection_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_question_selection_multiple_choice
task
:
bigbench_question_selection_multiple_choice
Prev
1
…
6
7
8
9
10
11
12
13
14
…
27
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment