Merge pull request #864 from EleutherAI/add-fewshot-config

[Refactor] CMMLU, C-Eval port ; Add fewshot config

Merge pull request #864 from EleutherAI/add-fewshot-config
[Refactor] CMMLU, C-Eval port ; Add fewshot config
bdddfec2 · Hailey Schoelkopf · GitHub · 0f6cd358 · f88ffeee · bdddfec2
Unverified Commit bdddfec2 authored Sep 21, 2023 by Hailey Schoelkopf Committed by GitHub Sep 21, 2023
11 changed files
--- a/lm_eval/tasks/cmmlu/cmmlu_default_professional_psychology.yaml
+++ b/lm_eval/tasks/cmmlu/cmmlu_default_professional_psychology.yaml
--- a/lm_eval/tasks/cmmlu/cmmlu_default_public_relations.yaml
+++ b/lm_eval/tasks/cmmlu/cmmlu_default_public_relations.yaml
--- a/lm_eval/tasks/cmmlu/cmmlu_default_security_study.yaml
+++ b/lm_eval/tasks/cmmlu/cmmlu_default_security_study.yaml
--- a/lm_eval/tasks/cmmlu/cmmlu_default_sociology.yaml
+++ b/lm_eval/tasks/cmmlu/cmmlu_default_sociology.yaml
--- a/lm_eval/tasks/cmmlu/cmmlu_default_sports_science.yaml
+++ b/lm_eval/tasks/cmmlu/cmmlu_default_sports_science.yaml
--- a/lm_eval/tasks/cmmlu/cmmlu_default_traditional_chinese_medicine.yaml
+++ b/lm_eval/tasks/cmmlu/cmmlu_default_traditional_chinese_medicine.yaml
--- a/lm_eval/tasks/cmmlu/cmmlu_default_virology.yaml
+++ b/lm_eval/tasks/cmmlu/cmmlu_default_virology.yaml
--- a/lm_eval/tasks/cmmlu/cmmlu_default_world_history.yaml
+++ b/lm_eval/tasks/cmmlu/cmmlu_default_world_history.yaml
--- a/lm_eval/tasks/cmmlu/cmmlu_default_world_religions.yaml
+++ b/lm_eval/tasks/cmmlu/cmmlu_default_world_religions.yaml
--- a/lm_eval/tasks/crows_pairs/README.md
+++ b/lm_eval/tasks/crows_pairs/README.md
--- a/lm_eval/tasks/storycloze/storycloze_2018.yaml
+++ b/lm_eval/tasks/storycloze/storycloze_2018.yaml