Bigbench fix (#1686)

* edit process multiple-choice * split template yaml * remove * modified multiple_choice tasks * udpate * Update multiple_choice_template_b_yaml * Update multiple_choice_template_a_yaml --------- Co-authored-by: Hailey Schoelkopf <65563625+haileyschoelkopf@users.noreply.github.com>

Bigbench fix (#1686)
* edit process multiple-choice * split template yaml * remove * modified multiple_choice tasks * udpate * Update multiple_choice_template_b_yaml * Update multiple_choice_template_a_yaml --------- Co-authored-by: Hailey Schoelkopf <65563625+haileyschoelkopf@users.noreply.github.com>
78a215e0 · Lintang Sutawika · GitHub · f2ea37e3 · 78a215e0 · f2ea37e3
Unverified Commit 78a215e0 authored May 25, 2024 by Lintang Sutawika Committed by GitHub May 24, 2024
20 changed files
--- a/lm_eval/tasks/bigbench/multiple_choice/presuppositions_as_nli.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/presuppositions_as_nli.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/qa_wikidata.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/qa_wikidata.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/question_selection.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/question_selection.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/real_or_fake_text.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/real_or_fake_text.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/reasoning_about_colored_objects.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/reasoning_about_colored_objects.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/repeat_copy_logic.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/repeat_copy_logic.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/rephrase.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/rephrase.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/riddle_sense.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/riddle_sense.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/ruin_names.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/ruin_names.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/salient_translation_error_detection.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/salient_translation_error_detection.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/scientific_press_release.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/scientific_press_release.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/semantic_parsing_in_context_sparc.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/semantic_parsing_in_context_sparc.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/semantic_parsing_spider.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/semantic_parsing_spider.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/sentence_ambiguity.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/sentence_ambiguity.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/similarities_abstraction.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/similarities_abstraction.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/simp_turing_concept.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/simp_turing_concept.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json_multiple_choice.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json_multiple_choice.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json_subtasks.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_json_subtasks.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_multiple_targets_json.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/simple_arithmetic_multiple_targets_json.yaml