Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
3e1301bb
"src/targets/vscode:/vscode.git/clone" did not exist on "e54db8dea0133f929560e610e10a4cdda8caef0b"
Commit
3e1301bb
authored
Jun 04, 2024
by
lintangsutawika
Browse files
resolved merge conflict from latest version
parents
fd9cd80f
070d31df
Changes
539
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
9 additions
and
53 deletions
+9
-53
lm_eval/tasks/bigbench/multiple_choice/real_or_fake_text.yaml
...val/tasks/bigbench/multiple_choice/real_or_fake_text.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/reasoning_about_colored_objects.yaml
...ench/multiple_choice/reasoning_about_colored_objects.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/repeat_copy_logic.yaml
...val/tasks/bigbench/multiple_choice/repeat_copy_logic.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/rephrase.yaml
lm_eval/tasks/bigbench/multiple_choice/rephrase.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/riddle_sense.yaml
lm_eval/tasks/bigbench/multiple_choice/riddle_sense.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/ruin_names.yaml
lm_eval/tasks/bigbench/multiple_choice/ruin_names.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/salient_translation_error_detection.yaml
.../multiple_choice/salient_translation_error_detection.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/scientific_press_release.yaml
...ks/bigbench/multiple_choice/scientific_press_release.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/semantic_parsing_in_context_sparc.yaml
...ch/multiple_choice/semantic_parsing_in_context_sparc.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/semantic_parsing_spider.yaml
...sks/bigbench/multiple_choice/semantic_parsing_spider.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/sentence_ambiguity.yaml
...al/tasks/bigbench/multiple_choice/sentence_ambiguity.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/similarities_abstraction.yaml
...ks/bigbench/multiple_choice/similarities_abstraction.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/simp_turing_concept.yaml
...l/tasks/bigbench/multiple_choice/simp_turing_concept.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json.yaml
...asks/bigbench/multiple_choice/simple_arithmetic_json.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json_multiple_choice.yaml
...ltiple_choice/simple_arithmetic_json_multiple_choice.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json_subtasks.yaml
...ench/multiple_choice/simple_arithmetic_json_subtasks.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_multiple_targets_json.yaml
...tiple_choice/simple_arithmetic_multiple_targets_json.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/simple_ethical_questions.yaml
...ks/bigbench/multiple_choice/simple_ethical_questions.yaml
+1
-1
lm_eval/tasks/bigbench/multiple_choice/simple_text_editing.yaml
...l/tasks/bigbench/multiple_choice/simple_text_editing.yaml
+0
-4
lm_eval/tasks/bigbench/multiple_choice/snarks.yaml
lm_eval/tasks/bigbench/multiple_choice/snarks.yaml
+1
-1
No files found.
lm_eval/tasks/bigbench/multiple_choice/real_or_fake_text.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
real_or_fake_text_zero_shot
dataset_name
:
real_or_fake_text_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_real_or_fake_text_multiple_choice
task
:
bigbench_real_or_fake_text_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/reasoning_about_colored_objects.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
reasoning_about_colored_objects_zero_shot
dataset_name
:
reasoning_about_colored_objects_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_reasoning_about_colored_objects_multiple_choice
task
:
bigbench_reasoning_about_colored_objects_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/repeat_copy_logic.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
repeat_copy_logic_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_repeat_copy_logic_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/rephrase.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
rephrase_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_rephrase_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/riddle_sense.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
riddle_sense_zero_shot
dataset_name
:
riddle_sense_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_riddle_sense_multiple_choice
task
:
bigbench_riddle_sense_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/ruin_names.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
ruin_names_zero_shot
dataset_name
:
ruin_names_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_ruin_names_multiple_choice
task
:
bigbench_ruin_names_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/salient_translation_error_detection.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
salient_translation_error_detection_zero_shot
dataset_name
:
salient_translation_error_detection_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_salient_translation_error_detection_multiple_choice
task
:
bigbench_salient_translation_error_detection_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/scientific_press_release.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
scientific_press_release_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_scientific_press_release_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/semantic_parsing_in_context_sparc.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
semantic_parsing_in_context_sparc_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_semantic_parsing_in_context_sparc_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/semantic_parsing_spider.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
semantic_parsing_spider_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_semantic_parsing_spider_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/sentence_ambiguity.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
sentence_ambiguity_zero_shot
dataset_name
:
sentence_ambiguity_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_sentence_ambiguity_multiple_choice
task
:
bigbench_sentence_ambiguity_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/similarities_abstraction.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
similarities_abstraction_zero_shot
dataset_name
:
similarities_abstraction_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_similarities_abstraction_multiple_choice
task
:
bigbench_similarities_abstraction_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/simp_turing_concept.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
simp_turing_concept_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_simp_turing_concept_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
simple_arithmetic_json_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_simple_arithmetic_json_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json_multiple_choice.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
simple_arithmetic_json_multiple_choice_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_simple_arithmetic_json_multiple_choice_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json_subtasks.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
simple_arithmetic_json_subtasks_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_simple_arithmetic_json_subtasks_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_multiple_targets_json.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
simple_arithmetic_multiple_targets_json_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_simple_arithmetic_multiple_targets_json_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/simple_ethical_questions.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
simple_ethical_questions_zero_shot
dataset_name
:
simple_ethical_questions_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_simple_ethical_questions_multiple_choice
task
:
bigbench_simple_ethical_questions_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/simple_text_editing.yaml
deleted
100644 → 0
View file @
fd9cd80f
# Generated by utils.py
dataset_name
:
simple_text_editing_zero_shot
include
:
../multiple_choice_template_yaml
task
:
bigbench_simple_text_editing_multiple_choice
lm_eval/tasks/bigbench/multiple_choice/snarks.yaml
View file @
3e1301bb
# Generated by utils.py
# Generated by utils.py
dataset_name
:
snarks_zero_shot
dataset_name
:
snarks_zero_shot
include
:
../multiple_choice_template_yaml
include
:
../multiple_choice_template_
a_
yaml
task
:
bigbench_snarks_multiple_choice
task
:
bigbench_snarks_multiple_choice
Prev
1
…
7
8
9
10
11
12
13
14
15
…
27
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment