Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
3e1301bb
Commit
3e1301bb
authored
Jun 04, 2024
by
lintangsutawika
Browse files
resolved merge conflict from latest version
parents
fd9cd80f
070d31df
Changes
539
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
12 additions
and
44 deletions
+12
-44
lm_eval/tasks/bigbench/multiple_choice/analogical_similarity.yaml
...tasks/bigbench/multiple_choice/analogical_similarity.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/analytic_entailment.yaml
...l/tasks/bigbench/multiple_choice/analytic_entailment.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/arithmetic.yaml
lm_eval/tasks/bigbench/multiple_choice/arithmetic.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/ascii_word_recognition.yaml
...asks/bigbench/multiple_choice/ascii_word_recognition.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/authorship_verification.yaml
...sks/bigbench/multiple_choice/authorship_verification.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/auto_categorization.yaml
...l/tasks/bigbench/multiple_choice/auto_categorization.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/auto_debugging.yaml
lm_eval/tasks/bigbench/multiple_choice/auto_debugging.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/bbq_lite_json.yaml
lm_eval/tasks/bigbench/multiple_choice/bbq_lite_json.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/bridging_anaphora_resolution_barqa.yaml
...h/multiple_choice/bridging_anaphora_resolution_barqa.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/causal_judgement.yaml
lm_eval/tasks/bigbench/multiple_choice/causal_judgement.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/causal_judgment.yaml
lm_eval/tasks/bigbench/multiple_choice/causal_judgment.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/cause_and_effect.yaml
lm_eval/tasks/bigbench/multiple_choice/cause_and_effect.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/checkmate_in_one.yaml
lm_eval/tasks/bigbench/multiple_choice/checkmate_in_one.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/chess_state_tracking.yaml
.../tasks/bigbench/multiple_choice/chess_state_tracking.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/chinese_remainder_theorem.yaml
...s/bigbench/multiple_choice/chinese_remainder_theorem.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/cifar10_classification.yaml
...asks/bigbench/multiple_choice/cifar10_classification.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/code_line_description.yaml
...tasks/bigbench/multiple_choice/code_line_description.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/codenames.yaml
lm_eval/tasks/bigbench/multiple_choice/codenames.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/color.yaml
lm_eval/tasks/bigbench/multiple_choice/color.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/common_morpheme.yaml
lm_eval/tasks/bigbench/multiple_choice/common_morpheme.yaml
+1
-1
No files found.
lm_eval/tasks/bigbench/multiple_choice/analogical_similarity.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
analogical_similarity_zero_shot
dataset_name
:
analogical_similarity_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_analogical_similarity_multiple_choice
task
:
bigbench_analogical_similarity_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/analytic_entailment.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
analytic_entailment_zero_shot
dataset_name
:
analytic_entailment_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_analytic_entailment_multiple_choice
task
:
bigbench_analytic_entailment_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/arithmetic.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
arithmetic_zero_shot
dataset_name
:
arithmetic_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_arithmetic_multiple_choice
task
:
bigbench_arithmetic_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/ascii_word_recognition.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
ascii_word_recognition_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_ascii_word_recognition_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/authorship_verification.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
authorship_verification_zero_shot
dataset_name
:
authorship_verification_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_authorship_verification_multiple_choice
task
:
bigbench_authorship_verification_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/auto_categorization.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
auto_categorization_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_auto_categorization_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/auto_debugging.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
auto_debugging_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_auto_debugging_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/bbq_lite_json.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
bbq_lite_json_zero_shot
dataset_name
:
bbq_lite_json_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_bbq_lite_json_multiple_choice
task
:
bigbench_bbq_lite_json_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/bridging_anaphora_resolution_barqa.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
bridging_anaphora_resolution_barqa_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_bridging_anaphora_resolution_barqa_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/causal_judgement.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
causal_judgment_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_causal_judgement_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/causal_judgment.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
causal_judgment_zero_shot
dataset_name
:
causal_judgment_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_causal_judgment_multiple_choice
task
:
bigbench_causal_judgment_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/cause_and_effect.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
cause_and_effect_zero_shot
dataset_name
:
cause_and_effect_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_cause_and_effect_multiple_choice
task
:
bigbench_cause_and_effect_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/checkmate_in_one.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
checkmate_in_one_zero_shot
dataset_name
:
checkmate_in_one_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_checkmate_in_one_multiple_choice
task
:
bigbench_checkmate_in_one_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/chess_state_tracking.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
chess_state_tracking_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_chess_state_tracking_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/chinese_remainder_theorem.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
chinese_remainder_theorem_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_chinese_remainder_theorem_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/cifar10_classification.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
cifar10_classification_zero_shot
dataset_name
:
cifar10_classification_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_cifar10_classification_multiple_choice
task
:
bigbench_cifar10_classification_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/code_line_description.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
code_line_description_zero_shot
dataset_name
:
code_line_description_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_code_line_description_multiple_choice
task
:
bigbench_code_line_description_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/codenames.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
codenames_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_codenames_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/color.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
color_zero_shot
dataset_name
:
color_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_color_multiple_choice
task
:
bigbench_color_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/common_morpheme.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
common_morpheme_zero_shot
dataset_name
:
common_morpheme_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_common_morpheme_multiple_choice
task
:
bigbench_common_morpheme_multiple_choice
Prev
1
2
3
4
5
6
7
8
9
…
27
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment