Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
c64bf9a9
Commit
c64bf9a9
authored
Oct 17, 2023
by
lintangsutawika
Browse files
change all mentions of `greedy_until` to `generate_until`
parent
04ca5671
Changes
236
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
80 additions
and
0 deletions
+80
-0
lm_eval/tasks/bigbench/generate_until/analogical_similarity.yaml
.../tasks/bigbench/generate_until/analogical_similarity.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/analytic_entailment.yaml
...al/tasks/bigbench/generate_until/analytic_entailment.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/arithmetic.yaml
lm_eval/tasks/bigbench/generate_until/arithmetic.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/ascii_word_recognition.yaml
...tasks/bigbench/generate_until/ascii_word_recognition.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/authorship_verification.yaml
...asks/bigbench/generate_until/authorship_verification.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/auto_categorization.yaml
...al/tasks/bigbench/generate_until/auto_categorization.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/auto_debugging.yaml
lm_eval/tasks/bigbench/generate_until/auto_debugging.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/bbq_lite_json.yaml
lm_eval/tasks/bigbench/generate_until/bbq_lite_json.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/bridging_anaphora_resolution_barqa.yaml
...ch/generate_until/bridging_anaphora_resolution_barqa.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/causal_judgment.yaml
lm_eval/tasks/bigbench/generate_until/causal_judgment.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/cause_and_effect.yaml
lm_eval/tasks/bigbench/generate_until/cause_and_effect.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/checkmate_in_one.yaml
lm_eval/tasks/bigbench/generate_until/checkmate_in_one.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/chess_state_tracking.yaml
...l/tasks/bigbench/generate_until/chess_state_tracking.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/chinese_remainder_theorem.yaml
...ks/bigbench/generate_until/chinese_remainder_theorem.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/cifar10_classification.yaml
...tasks/bigbench/generate_until/cifar10_classification.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/code_line_description.yaml
.../tasks/bigbench/generate_until/code_line_description.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/codenames.yaml
lm_eval/tasks/bigbench/generate_until/codenames.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/color.yaml
lm_eval/tasks/bigbench/generate_until/color.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/common_morpheme.yaml
lm_eval/tasks/bigbench/generate_until/common_morpheme.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/conceptual_combinations.yaml
...asks/bigbench/generate_until/conceptual_combinations.yaml
+4
-0
No files found.
lm_eval/tasks/bigbench/g
reedy
_until/analogical_similarity.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/analogical_similarity.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
analogical_similarity_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_analogical_similarity_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_analogical_similarity_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/analytic_entailment.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/analytic_entailment.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
analytic_entailment_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_analytic_entailment_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_analytic_entailment_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/arithmetic.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/arithmetic.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
arithmetic_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_arithmetic_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_arithmetic_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/ascii_word_recognition.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/ascii_word_recognition.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
ascii_word_recognition_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_ascii_word_recognition_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_ascii_word_recognition_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/authorship_verification.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/authorship_verification.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
authorship_verification_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_authorship_verification_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_authorship_verification_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/auto_categorization.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/auto_categorization.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
auto_categorization_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_auto_categorization_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_auto_categorization_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/auto_debugging.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/auto_debugging.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
auto_debugging_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_auto_debugging_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_auto_debugging_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/bbq_lite_json.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/bbq_lite_json.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
bbq_lite_json_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_bbq_lite_json_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_bbq_lite_json_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/bridging_anaphora_resolution_barqa.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/bridging_anaphora_resolution_barqa.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
bridging_anaphora_resolution_barqa_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_bridging_anaphora_resolution_barqa_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_bridging_anaphora_resolution_barqa_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/causal_judgment.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/causal_judgment.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
causal_judgment_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_causal_judgment_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_causal_judgment_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/cause_and_effect.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/cause_and_effect.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
cause_and_effect_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_cause_and_effect_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_cause_and_effect_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/checkmate_in_one.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/checkmate_in_one.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
checkmate_in_one_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_checkmate_in_one_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_checkmate_in_one_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/chess_state_tracking.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/chess_state_tracking.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
chess_state_tracking_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_chess_state_tracking_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_chess_state_tracking_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/chinese_remainder_theorem.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/chinese_remainder_theorem.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
chinese_remainder_theorem_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_chinese_remainder_theorem_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_chinese_remainder_theorem_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/cifar10_classification.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/cifar10_classification.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
cifar10_classification_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_cifar10_classification_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_cifar10_classification_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/code_line_description.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/code_line_description.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
code_line_description_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_code_line_description_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_code_line_description_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/codenames.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/codenames.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
codenames_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_codenames_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_codenames_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/color.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/color.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
color_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_color_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_color_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/common_morpheme.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/common_morpheme.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
common_morpheme_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_common_morpheme_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_common_morpheme_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/conceptual_combinations.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/conceptual_combinations.yaml
View file @
c64bf9a9
# Generated by utils.py
dataset_name
:
conceptual_combinations_zero_shot
include
:
../g
reedy
_until_template_yaml
task
:
bigbench_conceptual_combinations_g
reedy
_until
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_conceptual_combinations_g
enerate
_until
Prev
1
2
3
4
5
6
…
12
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment