Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
42df113c
Commit
42df113c
authored
Feb 09, 2021
by
Leo Gao
Browse files
Fix arithmetic
parent
84f14daf
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
1 deletion
+4
-1
lm_eval/tasks/arithmetic.py
lm_eval/tasks/arithmetic.py
+4
-1
No files found.
lm_eval/tasks/arithmetic.py
View file @
42df113c
...
...
@@ -56,7 +56,10 @@ class Arithmetic(Task):
return
doc
.
completion
def
load_doc
(
self
,
doc_json
):
return
ArithmeticDoc
(
context
=
doc_json
[
'context'
].
strip
(),
completion
=
doc_json
[
'completion'
].
strip
())
return
ArithmeticDoc
(
context
=
doc_json
[
'context'
].
strip
()
.
replace
(
'
\n\n
'
,
'
\n
'
)
.
replace
(
'Q:'
,
'Question:'
)
.
replace
(
'A:'
,
'Answer:'
),
completion
=
doc_json
[
'completion'
])
def
construct_requests
(
self
,
doc
,
ctx
):
ll
,
is_prediction
=
rf
.
loglikelihood
(
ctx
,
doc
.
completion
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment