Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
9a58cee6
Unverified
Commit
9a58cee6
authored
Jun 19, 2021
by
Jonathan Tow
Committed by
GitHub
Jun 19, 2021
Browse files
Grammar cleanup in the task docstring
parent
e5db2a1c
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
2 deletions
+2
-2
lm_eval/tasks/mc_taco.py
lm_eval/tasks/mc_taco.py
+2
-2
No files found.
lm_eval/tasks/mc_taco.py
View file @
9a58cee6
...
...
@@ -8,8 +8,8 @@ corresponding dataset is structured such that each multiple-choice-question gath
by the authors is split into question-option pairs, where each such pair gets
siloed into an individual document for plausibility testing. Because the harness
shuffles these documents, setting `--limit` will likely "cut off" certain candidate
answers. This is a problem because the task metrics require an exhaustive evaluation
of a question's options
(s
ee section 4 of the for details
paper)
.
answers. This is a problem because the task
's
metrics require an exhaustive evaluation
of a question's options
. S
ee section 4 of the
paper
for details.
@inproceedings{ZKNR19,
author = {Ben Zhou, Daniel Khashabi, Qiang Ning and Dan Roth},
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment