Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
c64bf9a9
Commit
c64bf9a9
authored
Oct 17, 2023
by
lintangsutawika
Browse files
change all mentions of `greedy_until` to `generate_until`
parent
04ca5671
Changes
236
Show whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
80 additions
and
0 deletions
+80
-0
lm_eval/tasks/bigbench/generate_until/conlang_translation.yaml
...al/tasks/bigbench/generate_until/conlang_translation.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/contextual_parametric_knowledge_conflicts.yaml
...rate_until/contextual_parametric_knowledge_conflicts.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/crash_blossom.yaml
lm_eval/tasks/bigbench/generate_until/crash_blossom.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/crass_ai.yaml
lm_eval/tasks/bigbench/generate_until/crass_ai.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/cryobiology_spanish.yaml
...al/tasks/bigbench/generate_until/cryobiology_spanish.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/cryptonite.yaml
lm_eval/tasks/bigbench/generate_until/cryptonite.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/cs_algorithms.yaml
lm_eval/tasks/bigbench/generate_until/cs_algorithms.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/dark_humor_detection.yaml
...l/tasks/bigbench/generate_until/dark_humor_detection.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/date_understanding.yaml
...val/tasks/bigbench/generate_until/date_understanding.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/disambiguation_qa.yaml
lm_eval/tasks/bigbench/generate_until/disambiguation_qa.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/discourse_marker_prediction.yaml
.../bigbench/generate_until/discourse_marker_prediction.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/disfl_qa.yaml
lm_eval/tasks/bigbench/generate_until/disfl_qa.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/dyck_languages.yaml
lm_eval/tasks/bigbench/generate_until/dyck_languages.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/elementary_math_qa.yaml
...val/tasks/bigbench/generate_until/elementary_math_qa.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/emoji_movie.yaml
lm_eval/tasks/bigbench/generate_until/emoji_movie.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/emojis_emotion_prediction.yaml
...ks/bigbench/generate_until/emojis_emotion_prediction.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/empirical_judgments.yaml
...al/tasks/bigbench/generate_until/empirical_judgments.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/english_proverbs.yaml
lm_eval/tasks/bigbench/generate_until/english_proverbs.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/english_russian_proverbs.yaml
...sks/bigbench/generate_until/english_russian_proverbs.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/entailed_polarity.yaml
lm_eval/tasks/bigbench/generate_until/entailed_polarity.yaml
+4
-0
No files found.
lm_eval/tasks/bigbench/g
reedy
_until/conlang_translation.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/conlang_translation.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
conlang_translation_zero_shot
dataset_name
:
conlang_translation_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_conlang_translation_g
reedy
_until
task
:
bigbench_conlang_translation_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/contextual_parametric_knowledge_conflicts.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/contextual_parametric_knowledge_conflicts.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
contextual_parametric_knowledge_conflicts_zero_shot
dataset_name
:
contextual_parametric_knowledge_conflicts_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_contextual_parametric_knowledge_conflicts_g
reedy
_until
task
:
bigbench_contextual_parametric_knowledge_conflicts_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/crash_blossom.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/crash_blossom.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
crash_blossom_zero_shot
dataset_name
:
crash_blossom_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_crash_blossom_g
reedy
_until
task
:
bigbench_crash_blossom_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/crass_ai.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/crass_ai.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
crass_ai_zero_shot
dataset_name
:
crass_ai_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_crass_ai_g
reedy
_until
task
:
bigbench_crass_ai_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/cryobiology_spanish.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/cryobiology_spanish.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
cryobiology_spanish_zero_shot
dataset_name
:
cryobiology_spanish_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_cryobiology_spanish_g
reedy
_until
task
:
bigbench_cryobiology_spanish_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/cryptonite.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/cryptonite.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
cryptonite_zero_shot
dataset_name
:
cryptonite_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_cryptonite_g
reedy
_until
task
:
bigbench_cryptonite_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/cs_algorithms.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/cs_algorithms.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
cs_algorithms_zero_shot
dataset_name
:
cs_algorithms_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_cs_algorithms_g
reedy
_until
task
:
bigbench_cs_algorithms_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/dark_humor_detection.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/dark_humor_detection.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
dark_humor_detection_zero_shot
dataset_name
:
dark_humor_detection_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_dark_humor_detection_g
reedy
_until
task
:
bigbench_dark_humor_detection_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/date_understanding.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/date_understanding.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
date_understanding_zero_shot
dataset_name
:
date_understanding_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_date_understanding_g
reedy
_until
task
:
bigbench_date_understanding_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/disambiguation_qa.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/disambiguation_qa.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
disambiguation_qa_zero_shot
dataset_name
:
disambiguation_qa_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_disambiguation_qa_g
reedy
_until
task
:
bigbench_disambiguation_qa_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/discourse_marker_prediction.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/discourse_marker_prediction.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
discourse_marker_prediction_zero_shot
dataset_name
:
discourse_marker_prediction_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_discourse_marker_prediction_g
reedy
_until
task
:
bigbench_discourse_marker_prediction_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/disfl_qa.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/disfl_qa.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
disfl_qa_zero_shot
dataset_name
:
disfl_qa_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_disfl_qa_g
reedy
_until
task
:
bigbench_disfl_qa_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/dyck_languages.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/dyck_languages.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
dyck_languages_zero_shot
dataset_name
:
dyck_languages_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_dyck_languages_g
reedy
_until
task
:
bigbench_dyck_languages_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/elementary_math_qa.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/elementary_math_qa.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
elementary_math_qa_zero_shot
dataset_name
:
elementary_math_qa_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_elementary_math_qa_g
reedy
_until
task
:
bigbench_elementary_math_qa_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/emoji_movie.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/emoji_movie.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
emoji_movie_zero_shot
dataset_name
:
emoji_movie_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_emoji_movie_g
reedy
_until
task
:
bigbench_emoji_movie_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/emojis_emotion_prediction.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/emojis_emotion_prediction.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
emojis_emotion_prediction_zero_shot
dataset_name
:
emojis_emotion_prediction_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_emojis_emotion_prediction_g
reedy
_until
task
:
bigbench_emojis_emotion_prediction_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/empirical_judgments.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/empirical_judgments.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
empirical_judgments_zero_shot
dataset_name
:
empirical_judgments_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_empirical_judgments_g
reedy
_until
task
:
bigbench_empirical_judgments_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/english_proverbs.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/english_proverbs.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
english_proverbs_zero_shot
dataset_name
:
english_proverbs_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_english_proverbs_g
reedy
_until
task
:
bigbench_english_proverbs_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/english_russian_proverbs.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/english_russian_proverbs.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
english_russian_proverbs_zero_shot
dataset_name
:
english_russian_proverbs_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_english_russian_proverbs_g
reedy
_until
task
:
bigbench_english_russian_proverbs_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/entailed_polarity.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/entailed_polarity.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
entailed_polarity_zero_shot
dataset_name
:
entailed_polarity_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_entailed_polarity_g
reedy
_until
task
:
bigbench_entailed_polarity_g
enerate
_until
Prev
1
2
3
4
5
6
7
…
12
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment