Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
b18c86f6
Commit
b18c86f6
authored
Jul 17, 2023
by
guijinSON
Browse files
Update csatqa.py
parent
f6afabd5
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
8 additions
and
1 deletion
+8
-1
lm_eval/tasks/csatqa.py
lm_eval/tasks/csatqa.py
+8
-1
No files found.
lm_eval/tasks/csatqa.py
View file @
b18c86f6
...
...
@@ -21,13 +21,20 @@ class CSATQA(MultipleChoiceTask):
choices
=
[
doc
[
"option#1"
],
doc
[
"option#2"
],
doc
[
"option#3"
],
doc
[
"option#4"
],
doc
[
"option#5"
]]
out_doc
=
{
"question"
:
doc
[
"question"
],
"context"
:
doc
[
"context"
],
"choices"
:
choices
,
"gold"
:
int
(
doc
[
'gold'
]),
}
return
out_doc
def
doc_to_text
(
self
,
doc
):
return
doc
[
"question"
]
instruction
=
f
"""다음을 읽고 정답으로 알맞은 것을 고르시요.
### Context:
{
doc
[
"context"
]
}
### Question:
{
doc
[
"question"
]
}
### Options:
(1)
{
doc
[
'choices'
][
0
]
}
\n
(2)
{
doc
[
'choices'
][
1
]
}
\n
(3)
{
doc
[
'choices'
][
2
]
}
\n
(4)
{
doc
[
'choices'
][
3
]
}
\n
(5)
{
doc
[
'choices'
][
4
]
}
### Answer: 주어진 문제의 정답은"""
return
instruction
class
WR
(
CSATQA
):
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment