Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
c64bf9a9
"tests/mbart/test_tokenization_mbart.py" did not exist on "cb276b41deaa97088e6d56fdca9ceb17db16ca19"
Commit
c64bf9a9
authored
Oct 17, 2023
by
lintangsutawika
Browse files
change all mentions of `greedy_until` to `generate_until`
parent
04ca5671
Changes
236
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
80 additions
and
0 deletions
+80
-0
lm_eval/tasks/bigbench/generate_until/reasoning_about_colored_objects.yaml
...bench/generate_until/reasoning_about_colored_objects.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/repeat_copy_logic.yaml
lm_eval/tasks/bigbench/generate_until/repeat_copy_logic.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/rephrase.yaml
lm_eval/tasks/bigbench/generate_until/rephrase.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/riddle_sense.yaml
lm_eval/tasks/bigbench/generate_until/riddle_sense.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/ruin_names.yaml
lm_eval/tasks/bigbench/generate_until/ruin_names.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/salient_translation_error_detection.yaml
...h/generate_until/salient_translation_error_detection.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/scientific_press_release.yaml
...sks/bigbench/generate_until/scientific_press_release.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/semantic_parsing_in_context_sparc.yaml
...nch/generate_until/semantic_parsing_in_context_sparc.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/semantic_parsing_spider.yaml
...asks/bigbench/generate_until/semantic_parsing_spider.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/sentence_ambiguity.yaml
...val/tasks/bigbench/generate_until/sentence_ambiguity.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/similarities_abstraction.yaml
...sks/bigbench/generate_until/similarities_abstraction.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/simp_turing_concept.yaml
...al/tasks/bigbench/generate_until/simp_turing_concept.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json.yaml
...tasks/bigbench/generate_until/simple_arithmetic_json.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_multiple_choice.yaml
...enerate_until/simple_arithmetic_json_multiple_choice.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/simple_arithmetic_json_subtasks.yaml
...bench/generate_until/simple_arithmetic_json_subtasks.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/simple_arithmetic_multiple_targets_json.yaml
...nerate_until/simple_arithmetic_multiple_targets_json.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/simple_ethical_questions.yaml
...sks/bigbench/generate_until/simple_ethical_questions.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/simple_text_editing.yaml
...al/tasks/bigbench/generate_until/simple_text_editing.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/snarks.yaml
lm_eval/tasks/bigbench/generate_until/snarks.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/social_iqa.yaml
lm_eval/tasks/bigbench/generate_until/social_iqa.yaml
+4
-0
No files found.
lm_eval/tasks/bigbench/g
reedy
_until/reasoning_about_colored_objects.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/reasoning_about_colored_objects.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
reasoning_about_colored_objects_zero_shot
dataset_name
:
reasoning_about_colored_objects_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_reasoning_about_colored_objects_g
reedy
_until
task
:
bigbench_reasoning_about_colored_objects_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/repeat_copy_logic.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/repeat_copy_logic.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
repeat_copy_logic_zero_shot
dataset_name
:
repeat_copy_logic_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_repeat_copy_logic_g
reedy
_until
task
:
bigbench_repeat_copy_logic_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/rephrase.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/rephrase.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
rephrase_zero_shot
dataset_name
:
rephrase_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_rephrase_g
reedy
_until
task
:
bigbench_rephrase_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/riddle_sense.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/riddle_sense.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
riddle_sense_zero_shot
dataset_name
:
riddle_sense_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_riddle_sense_g
reedy
_until
task
:
bigbench_riddle_sense_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/ruin_names.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/ruin_names.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
ruin_names_zero_shot
dataset_name
:
ruin_names_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_ruin_names_g
reedy
_until
task
:
bigbench_ruin_names_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/salient_translation_error_detection.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/salient_translation_error_detection.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
salient_translation_error_detection_zero_shot
dataset_name
:
salient_translation_error_detection_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_salient_translation_error_detection_g
reedy
_until
task
:
bigbench_salient_translation_error_detection_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/scientific_press_release.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/scientific_press_release.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
scientific_press_release_zero_shot
dataset_name
:
scientific_press_release_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_scientific_press_release_g
reedy
_until
task
:
bigbench_scientific_press_release_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/semantic_parsing_in_context_sparc.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/semantic_parsing_in_context_sparc.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
semantic_parsing_in_context_sparc_zero_shot
dataset_name
:
semantic_parsing_in_context_sparc_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_semantic_parsing_in_context_sparc_g
reedy
_until
task
:
bigbench_semantic_parsing_in_context_sparc_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/semantic_parsing_spider.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/semantic_parsing_spider.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
semantic_parsing_spider_zero_shot
dataset_name
:
semantic_parsing_spider_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_semantic_parsing_spider_g
reedy
_until
task
:
bigbench_semantic_parsing_spider_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/sentence_ambiguity.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/sentence_ambiguity.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
sentence_ambiguity_zero_shot
dataset_name
:
sentence_ambiguity_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_sentence_ambiguity_g
reedy
_until
task
:
bigbench_sentence_ambiguity_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/similarities_abstraction.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/similarities_abstraction.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
similarities_abstraction_zero_shot
dataset_name
:
similarities_abstraction_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_similarities_abstraction_g
reedy
_until
task
:
bigbench_similarities_abstraction_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/simp_turing_concept.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/simp_turing_concept.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
simp_turing_concept_zero_shot
dataset_name
:
simp_turing_concept_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_simp_turing_concept_g
reedy
_until
task
:
bigbench_simp_turing_concept_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/simple_arithmetic_json.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/simple_arithmetic_json.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
simple_arithmetic_json_zero_shot
dataset_name
:
simple_arithmetic_json_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_simple_arithmetic_json_g
reedy
_until
task
:
bigbench_simple_arithmetic_json_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/simple_arithmetic_json_multiple_choice.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/simple_arithmetic_json_multiple_choice.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
simple_arithmetic_json_multiple_choice_zero_shot
dataset_name
:
simple_arithmetic_json_multiple_choice_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_simple_arithmetic_json_multiple_choice_g
reedy
_until
task
:
bigbench_simple_arithmetic_json_multiple_choice_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/simple_arithmetic_json_subtasks.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/simple_arithmetic_json_subtasks.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
simple_arithmetic_json_subtasks_zero_shot
dataset_name
:
simple_arithmetic_json_subtasks_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_simple_arithmetic_json_subtasks_g
reedy
_until
task
:
bigbench_simple_arithmetic_json_subtasks_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/simple_arithmetic_multiple_targets_json.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/simple_arithmetic_multiple_targets_json.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
simple_arithmetic_multiple_targets_json_zero_shot
dataset_name
:
simple_arithmetic_multiple_targets_json_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_simple_arithmetic_multiple_targets_json_g
reedy
_until
task
:
bigbench_simple_arithmetic_multiple_targets_json_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/simple_ethical_questions.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/simple_ethical_questions.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
simple_ethical_questions_zero_shot
dataset_name
:
simple_ethical_questions_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_simple_ethical_questions_g
reedy
_until
task
:
bigbench_simple_ethical_questions_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/simple_text_editing.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/simple_text_editing.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
simple_text_editing_zero_shot
dataset_name
:
simple_text_editing_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_simple_text_editing_g
reedy
_until
task
:
bigbench_simple_text_editing_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/snarks.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/snarks.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
snarks_zero_shot
dataset_name
:
snarks_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_snarks_g
reedy
_until
task
:
bigbench_snarks_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/social_iqa.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/social_iqa.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
social_iqa_zero_shot
dataset_name
:
social_iqa_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_social_iqa_g
reedy
_until
task
:
bigbench_social_iqa_g
enerate
_until
Prev
1
…
4
5
6
7
8
9
10
11
12
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment