Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
e9610471
Unverified
Commit
e9610471
authored
Jul 27, 2023
by
GUIJIN SON
Committed by
GitHub
Jul 27, 2023
Browse files
Update csatqa.py
parent
602cf792
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
2 deletions
+2
-2
lm_eval/tasks/csatqa.py
lm_eval/tasks/csatqa.py
+2
-2
No files found.
lm_eval/tasks/csatqa.py
View file @
e9610471
...
...
@@ -22,14 +22,14 @@ class CSATQA(MultipleChoiceTask):
### Context:
{
doc
[
"context"
]
}
### Question:
{
doc
[
"question"
]
}
### Options:
(1)
{
doc
[
'option#1'
]
}
\n
(2)
{
doc
[
"option#2"
]
}
\n
(3)
{
doc
[
"option#3"
]
}
\n
(4)
{
doc
[
'option#4'
]
}
\n
(5)
{
doc
[
'option
$
5'
]
}
(1)
{
doc
[
'option#1'
]
}
\n
(2)
{
doc
[
"option#2"
]
}
\n
(3)
{
doc
[
"option#3"
]
}
\n
(4)
{
doc
[
'option#4'
]
}
\n
(5)
{
doc
[
'option
#
5'
]
}
### Answer: 주어진 문제의 정답은"""
choices
=
[
doc
[
"option#1"
],
doc
[
"option#2"
],
doc
[
"option#3"
],
doc
[
"option#4"
],
doc
[
"option#5"
]]
out_doc
=
{
"question"
:
instruction
,
"choices"
:
[
"(1)"
,
"(2)"
,
"(3)"
,
"(4)"
,
"(5)"
],
"gold"
:
int
(
doc
[
'gold'
]),
"gold"
:
int
(
doc
[
'gold'
])
-
1
,
}
return
out_doc
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment