Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
c64bf9a9
".github/vscode:/vscode.git/clone" did not exist on "0ffcccbca21ed0494348a6271cb64c853668de3d"
Commit
c64bf9a9
authored
Oct 17, 2023
by
lintangsutawika
Browse files
change all mentions of `greedy_until` to `generate_until`
parent
04ca5671
Changes
236
Show whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
80 additions
and
0 deletions
+80
-0
lm_eval/tasks/bigbench/generate_until/entailed_polarity_hindi.yaml
...asks/bigbench/generate_until/entailed_polarity_hindi.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/epistemic_reasoning.yaml
...al/tasks/bigbench/generate_until/epistemic_reasoning.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/evaluating_information_essentiality.yaml
...h/generate_until/evaluating_information_essentiality.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/fact_checker.yaml
lm_eval/tasks/bigbench/generate_until/fact_checker.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/fantasy_reasoning.yaml
lm_eval/tasks/bigbench/generate_until/fantasy_reasoning.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/few_shot_nlg.yaml
lm_eval/tasks/bigbench/generate_until/few_shot_nlg.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/figure_of_speech_detection.yaml
...s/bigbench/generate_until/figure_of_speech_detection.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/formal_fallacies_syllogisms_negation.yaml
.../generate_until/formal_fallacies_syllogisms_negation.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/gem.yaml
lm_eval/tasks/bigbench/generate_until/gem.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/gender_inclusive_sentences_german.yaml
...nch/generate_until/gender_inclusive_sentences_german.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/general_knowledge.yaml
lm_eval/tasks/bigbench/generate_until/general_knowledge.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/geometric_shapes.yaml
lm_eval/tasks/bigbench/generate_until/geometric_shapes.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/goal_step_wikihow.yaml
lm_eval/tasks/bigbench/generate_until/goal_step_wikihow.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/gre_reading_comprehension.yaml
...ks/bigbench/generate_until/gre_reading_comprehension.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/hhh_alignment.yaml
lm_eval/tasks/bigbench/generate_until/hhh_alignment.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/hindi_question_answering.yaml
...sks/bigbench/generate_until/hindi_question_answering.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/hindu_knowledge.yaml
lm_eval/tasks/bigbench/generate_until/hindu_knowledge.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/hinglish_toxicity.yaml
lm_eval/tasks/bigbench/generate_until/hinglish_toxicity.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/human_organs_senses.yaml
...al/tasks/bigbench/generate_until/human_organs_senses.yaml
+4
-0
lm_eval/tasks/bigbench/generate_until/hyperbaton.yaml
lm_eval/tasks/bigbench/generate_until/hyperbaton.yaml
+4
-0
No files found.
lm_eval/tasks/bigbench/g
reedy
_until/entailed_polarity_hindi.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/entailed_polarity_hindi.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
entailed_polarity_hindi_zero_shot
dataset_name
:
entailed_polarity_hindi_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_entailed_polarity_hindi_g
reedy
_until
task
:
bigbench_entailed_polarity_hindi_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/epistemic_reasoning.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/epistemic_reasoning.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
epistemic_reasoning_zero_shot
dataset_name
:
epistemic_reasoning_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_epistemic_reasoning_g
reedy
_until
task
:
bigbench_epistemic_reasoning_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/evaluating_information_essentiality.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/evaluating_information_essentiality.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
evaluating_information_essentiality_zero_shot
dataset_name
:
evaluating_information_essentiality_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_evaluating_information_essentiality_g
reedy
_until
task
:
bigbench_evaluating_information_essentiality_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/fact_checker.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/fact_checker.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
fact_checker_zero_shot
dataset_name
:
fact_checker_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_fact_checker_g
reedy
_until
task
:
bigbench_fact_checker_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/fantasy_reasoning.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/fantasy_reasoning.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
fantasy_reasoning_zero_shot
dataset_name
:
fantasy_reasoning_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_fantasy_reasoning_g
reedy
_until
task
:
bigbench_fantasy_reasoning_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/few_shot_nlg.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/few_shot_nlg.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
few_shot_nlg_zero_shot
dataset_name
:
few_shot_nlg_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_few_shot_nlg_g
reedy
_until
task
:
bigbench_few_shot_nlg_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/figure_of_speech_detection.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/figure_of_speech_detection.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
figure_of_speech_detection_zero_shot
dataset_name
:
figure_of_speech_detection_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_figure_of_speech_detection_g
reedy
_until
task
:
bigbench_figure_of_speech_detection_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/formal_fallacies_syllogisms_negation.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/formal_fallacies_syllogisms_negation.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
formal_fallacies_syllogisms_negation_zero_shot
dataset_name
:
formal_fallacies_syllogisms_negation_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_formal_fallacies_syllogisms_negation_g
reedy
_until
task
:
bigbench_formal_fallacies_syllogisms_negation_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/gem.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/gem.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
gem_zero_shot
dataset_name
:
gem_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_gem_g
reedy
_until
task
:
bigbench_gem_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/gender_inclusive_sentences_german.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/gender_inclusive_sentences_german.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
gender_inclusive_sentences_german_zero_shot
dataset_name
:
gender_inclusive_sentences_german_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_gender_inclusive_sentences_german_g
reedy
_until
task
:
bigbench_gender_inclusive_sentences_german_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/general_knowledge.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/general_knowledge.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
general_knowledge_zero_shot
dataset_name
:
general_knowledge_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_general_knowledge_g
reedy
_until
task
:
bigbench_general_knowledge_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/geometric_shapes.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/geometric_shapes.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
geometric_shapes_zero_shot
dataset_name
:
geometric_shapes_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_geometric_shapes_g
reedy
_until
task
:
bigbench_geometric_shapes_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/goal_step_wikihow.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/goal_step_wikihow.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
goal_step_wikihow_zero_shot
dataset_name
:
goal_step_wikihow_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_goal_step_wikihow_g
reedy
_until
task
:
bigbench_goal_step_wikihow_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/gre_reading_comprehension.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/gre_reading_comprehension.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
gre_reading_comprehension_zero_shot
dataset_name
:
gre_reading_comprehension_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_gre_reading_comprehension_g
reedy
_until
task
:
bigbench_gre_reading_comprehension_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/hhh_alignment.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/hhh_alignment.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
hhh_alignment_zero_shot
dataset_name
:
hhh_alignment_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_hhh_alignment_g
reedy
_until
task
:
bigbench_hhh_alignment_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/hindi_question_answering.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/hindi_question_answering.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
hindi_question_answering_zero_shot
dataset_name
:
hindi_question_answering_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_hindi_question_answering_g
reedy
_until
task
:
bigbench_hindi_question_answering_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/hindu_knowledge.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/hindu_knowledge.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
hindu_knowledge_zero_shot
dataset_name
:
hindu_knowledge_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_hindu_knowledge_g
reedy
_until
task
:
bigbench_hindu_knowledge_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/hinglish_toxicity.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/hinglish_toxicity.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
hinglish_toxicity_zero_shot
dataset_name
:
hinglish_toxicity_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_hinglish_toxicity_g
reedy
_until
task
:
bigbench_hinglish_toxicity_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/human_organs_senses.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/human_organs_senses.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
human_organs_senses_zero_shot
dataset_name
:
human_organs_senses_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_human_organs_senses_g
reedy
_until
task
:
bigbench_human_organs_senses_g
enerate
_until
lm_eval/tasks/bigbench/g
reedy
_until/hyperbaton.yaml
→
lm_eval/tasks/bigbench/g
enerate
_until/hyperbaton.yaml
View file @
c64bf9a9
# Generated by utils.py
# Generated by utils.py
dataset_name
:
hyperbaton_zero_shot
dataset_name
:
hyperbaton_zero_shot
include
:
../g
reedy
_until_template_yaml
include
:
../g
enerate
_until_template_yaml
task
:
bigbench_hyperbaton_g
reedy
_until
task
:
bigbench_hyperbaton_g
enerate
_until
Prev
1
2
3
4
5
6
7
8
…
12
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment