Bigbench fix (#1686)

* edit process multiple-choice * split template yaml * remove * modified multiple_choice tasks * udpate * Update multiple_choice_template_b_yaml * Update multiple_choice_template_a_yaml --------- Co-authored-by: Hailey Schoelkopf <65563625+haileyschoelkopf@users.noreply.github.com>

Bigbench fix (#1686)
* edit process multiple-choice * split template yaml * remove * modified multiple_choice tasks * udpate * Update multiple_choice_template_b_yaml * Update multiple_choice_template_a_yaml --------- Co-authored-by: Hailey Schoelkopf <65563625+haileyschoelkopf@users.noreply.github.com>
78a215e0 · Lintang Sutawika · GitHub · f2ea37e3 · 78a215e0 · f2ea37e3
Unverified Commit 78a215e0 authored May 25, 2024 by Lintang Sutawika Committed by GitHub May 24, 2024
20 changed files
--- a/lm_eval/tasks/bigbench/multiple_choice/multiemo.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/multiemo.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/natural_instructions.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/natural_instructions.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/navigate.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/navigate.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/nonsense_words_grammar.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/novel_concepts.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/object_counting.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/object_counting.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/odd_one_out.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/operators.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/operators.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/paragraph_segmentation.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/paragraph_segmentation.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/parsinlu_qa.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/parsinlu_reading_comprehension.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/parsinlu_reading_comprehension.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/penguins_in_a_table.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/periodic_elements.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/persian_idioms.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/phrase_relatedness.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/physical_intuition.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/physical_intuition.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/physics.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/physics.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/physics_questions.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/physics_questions.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/play_dialog_same_or_different.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/play_dialog_same_or_different.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/polish_sequence_labeling.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/polish_sequence_labeling.yaml