add humaneval+ and mbpp+ (#2734)

* add humaneval+ and mbpp+ * add newline at end of file

add humaneval+ and mbpp+ (#2734)
* add humaneval+ and mbpp+ * add newline at end of file
86bbf6ac · Minho Ryu · GitHub · 9b29da00 · 86bbf6ac · 86bbf6ac
Unverified Commit 86bbf6ac authored Feb 25, 2025 by Minho Ryu Committed by GitHub Feb 25, 2025
Hide whitespace changes
Inline Side-by-side

Showing with 8 additions and 0 deletions

lm_eval/tasks/humaneval/humaneval_plus.yaml lm_eval/tasks/humaneval/humaneval_plus.yaml +3 -0

lm_eval/tasks/mbpp/mbpp_plus.yaml lm_eval/tasks/mbpp/mbpp_plus.yaml +5 -0

No files found.
--- a/lm_eval/tasks/humaneval/humaneval_plus.yaml
+++ b/lm_eval/tasks/humaneval/humaneval_plus.yaml
+include: humaneval.yaml
+task: humaneval_plus
+dataset_path: evalplus/humanevalplus
--- a/lm_eval/tasks/mbpp/mbpp_plus.yaml
+++ b/lm_eval/tasks/mbpp/mbpp_plus.yaml
+include: mbpp.yaml
+task: mbpp_plus
+dataset_path: evalplus/mbppplus
+dataset_name: null
+doc_to_text: "You are an expert Python programmer, and here is your task: {{prompt if prompt is defined else text}} Your code should pass these tests:\n\n{{test_list[0]}}\n{{test_list[1]}}\n{{test_list[2]}}\n[BEGIN]\n"