Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
3f671a44
Commit
3f671a44
authored
Feb 04, 2025
by
Baber
Browse files
fix prompt
parent
41ecaba8
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
0 deletions
+4
-0
lm_eval/tasks/__init__.py
lm_eval/tasks/__init__.py
+4
-0
No files found.
lm_eval/tasks/__init__.py
View file @
3f671a44
...
...
@@ -22,9 +22,13 @@ def convert_mcq_to_generative(cfg: dict):
cfg
[
"output_type"
]
=
"generate_until"
cfg
[
"doc_to_text"
]
=
(
prompt
+
"
\n
"
+
cfg
.
get
(
"doc_to_text"
,
""
)
+
"
\n
"
+
'Your response should end with "The best answer is [the_answer_letter]" where the [the_answer_letter] is one of choice letters, A, B, C etc.'
)
cfg
[
"target_delimiter"
]
=
"
\n\n
"
cfg
[
"gen_prefix"
]
=
"The best answer is"
cfg
[
"generation_kwargs"
]
=
{
"until"
:
[
"."
],
"max_gen_toks"
:
10
}
cfg
[
"filter_list"
]
=
[
{
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment