Merge pull request #852 from EleutherAI/bigbench

[Refactor] bigbench

Merge pull request #852 from EleutherAI/bigbench
[Refactor] bigbench
7f2a9cbd · Hailey Schoelkopf · GitHub · c4fa6fb5 · 51a43a76 · 7f2a9cbd
Unverified Commit 7f2a9cbd authored Oct 04, 2023 by Hailey Schoelkopf Committed by GitHub Oct 04, 2023
19 changed files
--- a/lm_eval/tasks/bigbench/multiple_choice/symbol_interpretation.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/symbol_interpretation.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/temporal_sequences.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/temporal_sequences.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/tense.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/tense.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/timedial.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/timedial.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/topical_chat.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/topical_chat.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/tracking_shuffled_objects.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/tracking_shuffled_objects.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/understanding_fables.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/understanding_fables.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/undo_permutation.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/unit_conversion.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/unit_interpretation.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/unit_interpretation.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/unnatural_in_context_learning.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/unnatural_in_context_learning.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/vitaminc_fact_verification.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/vitaminc_fact_verification.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/what_is_the_tao.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/what_is_the_tao.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/which_wiki_edit.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/which_wiki_edit.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/winowhy.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/winowhy.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/word_sorting.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/word_sorting.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice/word_unscrambling.yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice/word_unscrambling.yaml
--- a/lm_eval/tasks/bigbench/multiple_choice_template_yaml
+++ b/lm_eval/tasks/bigbench/multiple_choice_template_yaml
--- a/lm_eval/tasks/bigbench/push_bigbench_dataset.py
+++ b/lm_eval/tasks/bigbench/push_bigbench_dataset.py