Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
c117e787
Commit
c117e787
authored
Jun 08, 2023
by
Oleh Shliazhko
Browse files
fix unreachable return
parent
f034974a
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
3 deletions
+2
-3
lm_eval/tasks/hendrycks_test.py
lm_eval/tasks/hendrycks_test.py
+2
-3
No files found.
lm_eval/tasks/hendrycks_test.py
View file @
c117e787
...
...
@@ -159,7 +159,6 @@ class GeneralHendrycksTest(MultipleChoiceTask):
"choices"
:
keys
,
"gold"
:
doc
[
"answer"
],
}
return
result
def
fewshot_examples
(
self
,
k
,
rnd
):
# fewshot_examples is not just sampling from train_docs because dev is
...
...
@@ -167,9 +166,9 @@ class GeneralHendrycksTest(MultipleChoiceTask):
if
self
.
_fewshot_docs
is
None
:
self
.
_fewshot_docs
=
list
(
map
(
self
.
_process_doc
,
self
.
dataset
[
"dev"
]))
# use the unchanged order of the dev set without sampling,
# use the unchanged order of the dev set without sampling,
# just as in the original code https://github.com/hendrycks/test/blob/master/evaluate.py#L28
return
self
.
_fewshot_docs
[:
k
]
return
self
.
_fewshot_docs
[:
k
]
def
doc_to_text
(
self
,
doc
):
return
doc
[
"query"
]
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment