Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
3e1301bb
Commit
3e1301bb
authored
Jun 04, 2024
by
lintangsutawika
Browse files
resolved merge conflict from latest version
parents
fd9cd80f
070d31df
Changes
539
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
13 additions
and
41 deletions
+13
-41
lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml
...asks/bigbench/multiple_choice/nonsense_words_grammar.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml
lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/object_counting.yaml
lm_eval/tasks/bigbench/multiple_choice/object_counting.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml
lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/operators.yaml
lm_eval/tasks/bigbench/multiple_choice/operators.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/paragraph_segmentation.yaml
...asks/bigbench/multiple_choice/paragraph_segmentation.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml
lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/parsinlu_reading_comprehension.yaml
...bench/multiple_choice/parsinlu_reading_comprehension.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml
...l/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml
...val/tasks/bigbench/multiple_choice/periodic_elements.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml
lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml
...al/tasks/bigbench/multiple_choice/phrase_relatedness.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/physical_intuition.yaml
...al/tasks/bigbench/multiple_choice/physical_intuition.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/physics.yaml
lm_eval/tasks/bigbench/multiple_choice/physics.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/physics_questions.yaml
...val/tasks/bigbench/multiple_choice/physics_questions.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/play_dialog_same_or_different.yaml
...gbench/multiple_choice/play_dialog_same_or_different.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/polish_sequence_labeling.yaml
...ks/bigbench/multiple_choice/polish_sequence_labeling.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/presuppositions_as_nli.yaml
...asks/bigbench/multiple_choice/presuppositions_as_nli.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/qa_wikidata.yaml
lm_eval/tasks/bigbench/multiple_choice/qa_wikidata.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/question_selection.yaml
...al/tasks/bigbench/multiple_choice/question_selection.yaml
+1
-1
No files found.
lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
nonsense_words_grammar_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_nonsense_words_grammar_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
novel_concepts_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_novel_concepts_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/object_counting.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
object_counting_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_object_counting_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
odd_one_out_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_odd_one_out_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/operators.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
operators_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_operators_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/paragraph_segmentation.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
paragraph_segmentation_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_paragraph_segmentation_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
parsinlu_qa_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_parsinlu_qa_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/parsinlu_reading_comprehension.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
parsinlu_reading_comprehension_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_parsinlu_reading_comprehension_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
penguins_in_a_table_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_penguins_in_a_table_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
periodic_elements_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_periodic_elements_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
persian_idioms_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_persian_idioms_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
phrase_relatedness_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_phrase_relatedness_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/physical_intuition.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
physical_intuition_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_physical_intuition_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/physics.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
physics_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_physics_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/physics_questions.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
physics_questions_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_physics_questions_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/play_dialog_same_or_different.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
play_dialog_same_or_different_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_play_dialog_same_or_different_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/polish_sequence_labeling.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
polish_sequence_labeling_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_polish_sequence_labeling_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/presuppositions_as_nli.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
presuppositions_as_nli_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_presuppositions_as_nli_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/qa_wikidata.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
qa_wikidata_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_qa_wikidata_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/question_selection.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
question_selection_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_question_selection_multiple_choice
Prev
1
…
6
7
8
9
10
11
12
13
14
…
27
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment