Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
3e1301bb
Commit
3e1301bb
authored
Jun 04, 2024
by
lintangsutawika
Browse files
resolved merge conflict from latest version
parents
fd9cd80f
070d31df
Changes
539
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
16 additions
and
32 deletions
+16
-32
lm_eval/tasks/bigbench/multiple_choice/social_iqa.yaml
lm_eval/tasks/bigbench/multiple_choice/social_iqa.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/social_support.yaml
lm_eval/tasks/bigbench/multiple_choice/social_support.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/sports_understanding.yaml
.../tasks/bigbench/multiple_choice/sports_understanding.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/strange_stories.yaml
lm_eval/tasks/bigbench/multiple_choice/strange_stories.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/strategyqa.yaml
lm_eval/tasks/bigbench/multiple_choice/strategyqa.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/sufficient_information.yaml
...asks/bigbench/multiple_choice/sufficient_information.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/suicide_risk.yaml
lm_eval/tasks/bigbench/multiple_choice/suicide_risk.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/swahili_english_proverbs.yaml
...ks/bigbench/multiple_choice/swahili_english_proverbs.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/swedish_to_german_proverbs.yaml
.../bigbench/multiple_choice/swedish_to_german_proverbs.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/symbol_interpretation.yaml
...tasks/bigbench/multiple_choice/symbol_interpretation.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/temporal_sequences.yaml
...al/tasks/bigbench/multiple_choice/temporal_sequences.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/tense.yaml
lm_eval/tasks/bigbench/multiple_choice/tense.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/timedial.yaml
lm_eval/tasks/bigbench/multiple_choice/timedial.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/topical_chat.yaml
lm_eval/tasks/bigbench/multiple_choice/topical_chat.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/tracking_shuffled_objects.yaml
...s/bigbench/multiple_choice/tracking_shuffled_objects.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/understanding_fables.yaml
.../tasks/bigbench/multiple_choice/understanding_fables.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml
lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml
lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/unit_interpretation.yaml
...l/tasks/bigbench/multiple_choice/unit_interpretation.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/unnatural_in_context_learning.yaml
...gbench/multiple_choice/unnatural_in_context_learning.yaml
+0
-4
No files found.
lm_eval/tasks/bigbench/multiple_choice/social_iqa.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
social_iqa_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_social_iqa_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/social_support.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
social_support_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_social_support_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/sports_understanding.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
sports_understanding_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_sports_understanding_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/strange_stories.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
strange_stories_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_strange_stories_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/strategyqa.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
strategyqa_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
b_
yaml
task
:
bigbench_strategyqa_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/sufficient_information.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
sufficient_information_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_sufficient_information_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/suicide_risk.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
suicide_risk_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_suicide_risk_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/swahili_english_proverbs.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
swahili_english_proverbs_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_swahili_english_proverbs_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/swedish_to_german_proverbs.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
swedish_to_german_proverbs_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_swedish_to_german_proverbs_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/symbol_interpretation.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
symbol_interpretation_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_symbol_interpretation_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/temporal_sequences.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
temporal_sequences_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_temporal_sequences_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/tense.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
tense_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_tense_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/timedial.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
timedial_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_timedial_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/topical_chat.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
topical_chat_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_topical_chat_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/tracking_shuffled_objects.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
tracking_shuffled_objects_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_tracking_shuffled_objects_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/understanding_fables.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
understanding_fables_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_understanding_fables_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
undo_permutation_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_undo_permutation_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
unit_conversion_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_unit_conversion_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/unit_interpretation.yaml
View file @
3e1301bb
# Generated by utils.py
dataset_name
:
unit_interpretation_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_unit_interpretation_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/unnatural_in_context_learning.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
unnatural_in_context_learning_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_unnatural_in_context_learning_multiple_choice
Prev
1
…
8
9
10
11
12
13
14
15
16
…
27
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment