Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
ec656bb1
Commit
ec656bb1
authored
Dec 13, 2023
by
lintangsutawika
Browse files
format
parent
6f4f9e1c
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
6 deletions
+2
-6
lm_eval/tasks/arithmetic/alternative_worlds/utils.py
lm_eval/tasks/arithmetic/alternative_worlds/utils.py
+2
-6
No files found.
lm_eval/tasks/arithmetic/alternative_worlds/utils.py
View file @
ec656bb1
...
...
@@ -2,26 +2,24 @@ import re
# Original Prompt
# Question: What is (9 + 8) * 2? Answer:
def
style_00
(
docs
):
def
style_00
(
docs
):
# What is (9 + 8) * 2?
return
docs
[
"context"
]
def
style_01
(
docs
):
# What is (9 + 8) * 2?
return
docs
[
"context"
].
replace
(
"Question: "
,
""
).
replace
(
" Answer:"
,
""
)
def
style_02
(
docs
):
# Q: What is (9 + 8) * 2? A:
return
docs
[
"context"
].
replace
(
"Question: "
,
"Q: "
).
replace
(
" Answer:"
,
" A:"
)
def
style_03
(
docs
):
# Solve (9 + 8) * 2.
return
(
docs
[
"context"
].
replace
(
"Question: What is"
,
"Solve"
).
replace
(
" Answer:"
,
"."
)
...
...
@@ -29,12 +27,10 @@ def style_03(docs):
def
style_04
(
docs
):
# (9 + 8) * 2 =
return
docs
[
"context"
].
replace
(
"Question: What is "
,
""
).
replace
(
" Answer:"
,
" ="
)
def
style_05
(
docs
):
# What is (9 + 8) * 2? Answer:
return
docs
[
"context"
].
replace
(
"Question: "
,
""
)
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment