Merge pull request #816 from EleutherAI/flan-benchmark

[Refactor] Flan benchmark

Merge pull request #816 from EleutherAI/flan-benchmark
[Refactor] Flan benchmark
6769119f · Hailey Schoelkopf · GitHub · 4824a832 · 7d5e511c · 6769119f
Unverified Commit 6769119f authored Oct 06, 2023 by Hailey Schoelkopf Committed by GitHub Oct 06, 2023
8 changed files
--- a/lm_eval/tasks/super_glue/cb/t5_utils.py
+++ b/lm_eval/tasks/super_glue/cb/t5_utils.py
--- a/lm_eval/tasks/super_glue/copa/t5-prompt.yaml
+++ b/lm_eval/tasks/super_glue/copa/t5-prompt.yaml
--- a/lm_eval/tasks/super_glue/multirc/t5-prompt.yaml
+++ b/lm_eval/tasks/super_glue/multirc/t5-prompt.yaml
--- a/lm_eval/tasks/super_glue/record/t5-prompt.yaml
+++ b/lm_eval/tasks/super_glue/record/t5-prompt.yaml
--- a/lm_eval/tasks/super_glue/rte/t5-prompt.yaml
+++ b/lm_eval/tasks/super_glue/rte/t5-prompt.yaml
--- a/lm_eval/tasks/super_glue/wic/t5-prompt.yaml
+++ b/lm_eval/tasks/super_glue/wic/t5-prompt.yaml
--- a/lm_eval/tasks/super_glue/wsc/t5-prompt.yaml
+++ b/lm_eval/tasks/super_glue/wsc/t5-prompt.yaml
--- a/lm_eval/utils.py
+++ b/lm_eval/utils.py