Merge branch 'big-refactor' of...

Merge branch 'big-refactor' of https://github.com/EleutherAI/lm-evaluation-harness into openai_completions

Merge branch 'big-refactor' of...
Merge branch 'big-refactor' of https://github.com/EleutherAI/lm-evaluation-harness into openai_completions
5d3bf2e7 · lintangsutawika · f66730c4 · bf26d979 · 5d3bf2e7 · 5d3bf2e7
Commit 5d3bf2e7 authored Nov 28, 2023 by lintangsutawika
20 changed files
--- a/lm_eval/tasks/bbh/flan_zeroshot/date_understanding.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/date_understanding.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/disambiguation_qa.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/disambiguation_qa.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/dyck_languages.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/dyck_languages.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/formal_fallacies.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/formal_fallacies.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/geometric_shapes.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/geometric_shapes.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/hyperbaton.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/hyperbaton.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/logical_deduction_five_objects.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/logical_deduction_five_objects.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/logical_deduction_seven_objects.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/logical_deduction_seven_objects.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/logical_deduction_three_objects.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/logical_deduction_three_objects.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/movie_recommendation.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/movie_recommendation.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/multistep_arithmetic_two.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/multistep_arithmetic_two.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/navigate.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/navigate.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/object_counting.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/object_counting.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/penguins_in_a_table.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/penguins_in_a_table.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/reasoning_about_colored_objects.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/reasoning_about_colored_objects.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/ruin_names.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/ruin_names.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/salient_translation_error_detection.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/salient_translation_error_detection.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/snarks.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/snarks.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/sports_understanding.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/sports_understanding.yaml
--- a/lm_eval/tasks/bbh/flan_zeroshot/temporal_sequences.yaml
+++ b/lm_eval/tasks/bbh/flan_zeroshot/temporal_sequences.yaml