Merge branch 'big-refactor' of...

Merge branch 'big-refactor' of https://github.com/EleutherAI/lm-evaluation-harness into mmlu_subgroups

Merge branch 'big-refactor' of...
Merge branch 'big-refactor' of https://github.com/EleutherAI/lm-evaluation-harness into mmlu_subgroups
f77a3a27 · lintangsutawika · 109ed1c7 · f8342178 · f77a3a27 · f77a3a27
Commit f77a3a27 authored Oct 17, 2023 by lintangsutawika
20 changed files
--- a/lm_eval/tasks/bigbench/greedy_until/unit_conversion.yaml
+++ b/lm_eval/tasks/bigbench/greedy_until/unit_conversion.yaml
--- a/lm_eval/tasks/bigbench/greedy_until/unit_interpretation.yaml
+++ b/lm_eval/tasks/bigbench/greedy_until/unit_interpretation.yaml
--- a/lm_eval/tasks/bigbench/greedy_until/unnatural_in_context_learning.yaml
+++ b/lm_eval/tasks/bigbench/greedy_until/unnatural_in_context_learning.yaml
--- a/lm_eval/tasks/bigbench/greedy_until/vitaminc_fact_verification.yaml
+++ b/lm_eval/tasks/bigbench/greedy_until/vitaminc_fact_verification.yaml
--- a/lm_eval/tasks/bigbench/greedy_until/what_is_the_tao.yaml
+++ b/lm_eval/tasks/bigbench/greedy_until/what_is_the_tao.yaml
--- a/lm_eval/tasks/bigbench/greedy_until/which_wiki_edit.yaml
+++ b/lm_eval/tasks/bigbench/greedy_until/which_wiki_edit.yaml
--- a/lm_eval/tasks/bigbench/greedy_until/winowhy.yaml
+++ b/lm_eval/tasks/bigbench/greedy_until/winowhy.yaml
--- a/lm_eval/tasks/bigbench/greedy_until/word_sorting.yaml
+++ b/lm_eval/tasks/bigbench/greedy_until/word_sorting.yaml
--- a/lm_eval/tasks/bigbench/greedy_until/word_unscrambling.yaml
+++ b/lm_eval/tasks/bigbench/greedy_until/word_unscrambling.yaml
--- a/lm_eval/tasks/bigbench/greedy_until_template_yaml
+++ b/lm_eval/tasks/bigbench/greedy_until_template_yaml
--- a/lm_eval/tasks/code_x_glue/code-text/go.yaml
+++ b/lm_eval/tasks/code_x_glue/code-text/go.yaml
--- a/lm_eval/tasks/code_x_glue/code-text/java.yaml
+++ b/lm_eval/tasks/code_x_glue/code-text/java.yaml
--- a/lm_eval/tasks/code_x_glue/code-text/javascript.yaml
+++ b/lm_eval/tasks/code_x_glue/code-text/javascript.yaml
--- a/lm_eval/tasks/code_x_glue/code-text/php.yaml
+++ b/lm_eval/tasks/code_x_glue/code-text/php.yaml
--- a/lm_eval/tasks/code_x_glue/code-text/python.yaml
+++ b/lm_eval/tasks/code_x_glue/code-text/python.yaml
--- a/lm_eval/tasks/code_x_glue/code-text/ruby.yaml
+++ b/lm_eval/tasks/code_x_glue/code-text/ruby.yaml
--- a/lm_eval/tasks/coqa/default.yaml
+++ b/lm_eval/tasks/coqa/default.yaml
--- a/lm_eval/tasks/drop/default.yaml
+++ b/lm_eval/tasks/drop/default.yaml
--- a/lm_eval/tasks/gsm8k/gsm8k-cot.yaml
+++ b/lm_eval/tasks/gsm8k/gsm8k-cot.yaml
--- a/lm_eval/tasks/gsm8k/gsm8k.yaml
+++ b/lm_eval/tasks/gsm8k/gsm8k.yaml