Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
8806944e
Commit
8806944e
authored
Oct 03, 2023
by
haileyschoelkopf
Browse files
push updated bigbench task yamls
parent
1a77b4d5
Changes
334
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
20 additions
and
20 deletions
+20
-20
lm_eval/tasks/bigbench/greedy_until/question_selection.yaml
lm_eval/tasks/bigbench/greedy_until/question_selection.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/real_or_fake_text.yaml
lm_eval/tasks/bigbench/greedy_until/real_or_fake_text.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/reasoning_about_colored_objects.yaml
...igbench/greedy_until/reasoning_about_colored_objects.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/repeat_copy_logic.yaml
lm_eval/tasks/bigbench/greedy_until/repeat_copy_logic.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/rephrase.yaml
lm_eval/tasks/bigbench/greedy_until/rephrase.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/riddle_sense.yaml
lm_eval/tasks/bigbench/greedy_until/riddle_sense.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/ruin_names.yaml
lm_eval/tasks/bigbench/greedy_until/ruin_names.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/salient_translation_error_detection.yaml
...nch/greedy_until/salient_translation_error_detection.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/scientific_press_release.yaml
...tasks/bigbench/greedy_until/scientific_press_release.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/semantic_parsing_in_context_sparc.yaml
...bench/greedy_until/semantic_parsing_in_context_sparc.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/semantic_parsing_spider.yaml
.../tasks/bigbench/greedy_until/semantic_parsing_spider.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/sentence_ambiguity.yaml
lm_eval/tasks/bigbench/greedy_until/sentence_ambiguity.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/similarities_abstraction.yaml
...tasks/bigbench/greedy_until/similarities_abstraction.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/simp_turing_concept.yaml
lm_eval/tasks/bigbench/greedy_until/simp_turing_concept.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/simple_arithmetic_json.yaml
...l/tasks/bigbench/greedy_until/simple_arithmetic_json.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/simple_arithmetic_json_multiple_choice.yaml
.../greedy_until/simple_arithmetic_json_multiple_choice.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/simple_arithmetic_json_subtasks.yaml
...igbench/greedy_until/simple_arithmetic_json_subtasks.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/simple_arithmetic_multiple_targets_json.yaml
...greedy_until/simple_arithmetic_multiple_targets_json.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/simple_ethical_questions.yaml
...tasks/bigbench/greedy_until/simple_ethical_questions.yaml
+1
-1
lm_eval/tasks/bigbench/greedy_until/simple_text_editing.yaml
lm_eval/tasks/bigbench/greedy_until/simple_text_editing.yaml
+1
-1
No files found.
lm_eval/tasks/bigbench/greedy_until/question_selection.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
question_selection
dataset_name
:
question_selection
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_question_selection_greedy_until
task
:
bigbench_question_selection_greedy_until
lm_eval/tasks/bigbench/greedy_until/real_or_fake_text.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
real_or_fake_text
dataset_name
:
real_or_fake_text
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_real_or_fake_text_greedy_until
task
:
bigbench_real_or_fake_text_greedy_until
lm_eval/tasks/bigbench/greedy_until/reasoning_about_colored_objects.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
reasoning_about_colored_objects
dataset_name
:
reasoning_about_colored_objects
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_reasoning_about_colored_objects_greedy_until
task
:
bigbench_reasoning_about_colored_objects_greedy_until
lm_eval/tasks/bigbench/greedy_until/repeat_copy_logic.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
repeat_copy_logic
dataset_name
:
repeat_copy_logic
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_repeat_copy_logic_greedy_until
task
:
bigbench_repeat_copy_logic_greedy_until
lm_eval/tasks/bigbench/greedy_until/rephrase.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
rephrase
dataset_name
:
rephrase
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_rephrase_greedy_until
task
:
bigbench_rephrase_greedy_until
lm_eval/tasks/bigbench/greedy_until/riddle_sense.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
riddle_sense
dataset_name
:
riddle_sense
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_riddle_sense_greedy_until
task
:
bigbench_riddle_sense_greedy_until
lm_eval/tasks/bigbench/greedy_until/ruin_names.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
ruin_names
dataset_name
:
ruin_names
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_ruin_names_greedy_until
task
:
bigbench_ruin_names_greedy_until
lm_eval/tasks/bigbench/greedy_until/salient_translation_error_detection.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
salient_translation_error_detection
dataset_name
:
salient_translation_error_detection
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_salient_translation_error_detection_greedy_until
task
:
bigbench_salient_translation_error_detection_greedy_until
lm_eval/tasks/bigbench/greedy_until/scientific_press_release.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
scientific_press_release
dataset_name
:
scientific_press_release
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_scientific_press_release_greedy_until
task
:
bigbench_scientific_press_release_greedy_until
lm_eval/tasks/bigbench/greedy_until/semantic_parsing_in_context_sparc.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
semantic_parsing_in_context_sparc
dataset_name
:
semantic_parsing_in_context_sparc
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_semantic_parsing_in_context_sparc_greedy_until
task
:
bigbench_semantic_parsing_in_context_sparc_greedy_until
lm_eval/tasks/bigbench/greedy_until/semantic_parsing_spider.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
semantic_parsing_spider
dataset_name
:
semantic_parsing_spider
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_semantic_parsing_spider_greedy_until
task
:
bigbench_semantic_parsing_spider_greedy_until
lm_eval/tasks/bigbench/greedy_until/sentence_ambiguity.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
sentence_ambiguity
dataset_name
:
sentence_ambiguity
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_sentence_ambiguity_greedy_until
task
:
bigbench_sentence_ambiguity_greedy_until
lm_eval/tasks/bigbench/greedy_until/similarities_abstraction.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
similarities_abstraction
dataset_name
:
similarities_abstraction
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_similarities_abstraction_greedy_until
task
:
bigbench_similarities_abstraction_greedy_until
lm_eval/tasks/bigbench/greedy_until/simp_turing_concept.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
simp_turing_concept
dataset_name
:
simp_turing_concept
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_simp_turing_concept_greedy_until
task
:
bigbench_simp_turing_concept_greedy_until
lm_eval/tasks/bigbench/greedy_until/simple_arithmetic_json.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
simple_arithmetic_json
dataset_name
:
simple_arithmetic_json
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_simple_arithmetic_json_greedy_until
task
:
bigbench_simple_arithmetic_json_greedy_until
lm_eval/tasks/bigbench/greedy_until/simple_arithmetic_json_multiple_choice.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
simple_arithmetic_json_multiple_choice
dataset_name
:
simple_arithmetic_json_multiple_choice
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_simple_arithmetic_json_multiple_choice_greedy_until
task
:
bigbench_simple_arithmetic_json_multiple_choice_greedy_until
lm_eval/tasks/bigbench/greedy_until/simple_arithmetic_json_subtasks.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
simple_arithmetic_json_subtasks
dataset_name
:
simple_arithmetic_json_subtasks
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_simple_arithmetic_json_subtasks_greedy_until
task
:
bigbench_simple_arithmetic_json_subtasks_greedy_until
lm_eval/tasks/bigbench/greedy_until/simple_arithmetic_multiple_targets_json.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
simple_arithmetic_multiple_targets_json
dataset_name
:
simple_arithmetic_multiple_targets_json
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_simple_arithmetic_multiple_targets_json_greedy_until
task
:
bigbench_simple_arithmetic_multiple_targets_json_greedy_until
lm_eval/tasks/bigbench/greedy_until/simple_ethical_questions.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
simple_ethical_questions
dataset_name
:
simple_ethical_questions
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_simple_ethical_questions_greedy_until
task
:
bigbench_simple_ethical_questions_greedy_until
lm_eval/tasks/bigbench/greedy_until/simple_text_editing.yaml
View file @
8806944e
# Generated by utils.py
# Generated by utils.py
dataset_name
:
simple_text_editing
dataset_name
:
simple_text_editing
_zero_shot
include
:
../greedy_until_template_yaml
include
:
../greedy_until_template_yaml
task
:
bigbench_simple_text_editing_greedy_until
task
:
bigbench_simple_text_editing_greedy_until
Prev
1
…
3
4
5
6
7
8
9
10
11
…
17
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment