Skip to content

GitLab

"vscode:/vscode.git/clone" did not exist on "edf9014c5b56cf27f2d4ab4250bfb4f3390a1608"
Unverified Commit 9a58cee6 authored by Jonathan Tow's avatar Jonathan Tow Committed by GitHub
Browse files

Grammar cleanup in the task docstring

parent e5db2a1c
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
lm_eval/tasks/mc_taco.py
View file @ 9a58cee6
...@@ -8,8 +8,8 @@ corresponding dataset is structured such that each multiple-choice-question gath ...@@ -8,8 +8,8 @@ corresponding dataset is structured such that each multiple-choice-question gath
by the authors is split into question-option pairs, where each such pair gets by the authors is split into question-option pairs, where each such pair gets
siloed into an individual document for plausibility testing. Because the harness siloed into an individual document for plausibility testing. Because the harness
shuffles these documents, setting `--limit` will likely "cut off" certain candidate shuffles these documents, setting `--limit` will likely "cut off" certain candidate
answers. This is a problem because the task metrics require an exhaustive evaluation answers. This is a problem because the task's metrics require an exhaustive evaluation
of a question's options (see section 4 of the for details paper). of a question's options. See section 4 of the paper for details.
@inproceedings{ZKNR19, @inproceedings{ZKNR19,
author = {Ben Zhou, Daniel Khashabi, Qiang Ning and Dan Roth}, author = {Ben Zhou, Daniel Khashabi, Qiang Ning and Dan Roth},
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment