Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
c74e2761
Commit
c74e2761
authored
Dec 06, 2023
by
lintangsutawika
Browse files
reformat
parent
cc572624
Changes
155
Show whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
31 additions
and
24 deletions
+31
-24
lm_eval/tasks/arithmetic/alternative_worlds/style_04/arithmetic_3ds.yaml
...rithmetic/alternative_worlds/style_04/arithmetic_3ds.yaml
+1
-1
lm_eval/tasks/arithmetic/alternative_worlds/style_04/arithmetic_4da.yaml
...rithmetic/alternative_worlds/style_04/arithmetic_4da.yaml
+1
-1
lm_eval/tasks/arithmetic/alternative_worlds/style_04/arithmetic_4ds.yaml
...rithmetic/alternative_worlds/style_04/arithmetic_4ds.yaml
+1
-1
lm_eval/tasks/arithmetic/alternative_worlds/style_04/arithmetic_5da.yaml
...rithmetic/alternative_worlds/style_04/arithmetic_5da.yaml
+1
-1
lm_eval/tasks/arithmetic/alternative_worlds/style_04/arithmetic_5ds.yaml
...rithmetic/alternative_worlds/style_04/arithmetic_5ds.yaml
+1
-1
lm_eval/tasks/arithmetic/alternative_worlds/style_05/_template_05_yaml
.../arithmetic/alternative_worlds/style_05/_template_05_yaml
+1
-1
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_1dc.yaml
...rithmetic/alternative_worlds/style_05/arithmetic_1dc.yaml
+1
-1
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_2da.yaml
...rithmetic/alternative_worlds/style_05/arithmetic_2da.yaml
+1
-1
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_2dm.yaml
...rithmetic/alternative_worlds/style_05/arithmetic_2dm.yaml
+1
-1
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_2ds.yaml
...rithmetic/alternative_worlds/style_05/arithmetic_2ds.yaml
+1
-1
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_3da.yaml
...rithmetic/alternative_worlds/style_05/arithmetic_3da.yaml
+1
-1
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_3ds.yaml
...rithmetic/alternative_worlds/style_05/arithmetic_3ds.yaml
+1
-1
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_4da.yaml
...rithmetic/alternative_worlds/style_05/arithmetic_4da.yaml
+1
-1
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_4ds.yaml
...rithmetic/alternative_worlds/style_05/arithmetic_4ds.yaml
+1
-1
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_5da.yaml
...rithmetic/alternative_worlds/style_05/arithmetic_5da.yaml
+1
-1
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_5ds.yaml
...rithmetic/alternative_worlds/style_05/arithmetic_5ds.yaml
+1
-1
lm_eval/tasks/arithmetic/alternative_worlds/utils.py
lm_eval/tasks/arithmetic/alternative_worlds/utils.py
+9
-2
lm_eval/tasks/hellaswag/alternative_worlds/README.md
lm_eval/tasks/hellaswag/alternative_worlds/README.md
+2
-2
lm_eval/tasks/hellaswag/alternative_worlds/style_01/a.yaml
lm_eval/tasks/hellaswag/alternative_worlds/style_01/a.yaml
+2
-2
lm_eval/tasks/hellaswag/alternative_worlds/style_01/b.yaml
lm_eval/tasks/hellaswag/alternative_worlds/style_01/b.yaml
+2
-2
No files found.
lm_eval/tasks/arithmetic/alternative_worlds/style_04/arithmetic_3ds.yaml
View file @
c74e2761
lm_eval/tasks/arithmetic/alternative_worlds/style_04/arithmetic_4da.yaml
View file @
c74e2761
lm_eval/tasks/arithmetic/alternative_worlds/style_04/arithmetic_4ds.yaml
View file @
c74e2761
lm_eval/tasks/arithmetic/alternative_worlds/style_04/arithmetic_5da.yaml
View file @
c74e2761
lm_eval/tasks/arithmetic/alternative_worlds/style_04/arithmetic_5ds.yaml
View file @
c74e2761
lm_eval/tasks/arithmetic/alternative_worlds/style_05/_template_05_yaml
View file @
c74e2761
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_1dc.yaml
View file @
c74e2761
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_2da.yaml
View file @
c74e2761
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_2dm.yaml
View file @
c74e2761
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_2ds.yaml
View file @
c74e2761
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_3da.yaml
View file @
c74e2761
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_3ds.yaml
View file @
c74e2761
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_4da.yaml
View file @
c74e2761
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_4ds.yaml
View file @
c74e2761
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_5da.yaml
View file @
c74e2761
lm_eval/tasks/arithmetic/alternative_worlds/style_05/arithmetic_5ds.yaml
View file @
c74e2761
lm_eval/tasks/arithmetic/alternative_worlds/utils.py
View file @
c74e2761
...
@@ -7,26 +7,33 @@ def style_00(docs):
...
@@ -7,26 +7,33 @@ def style_00(docs):
# What is (9 + 8) * 2?
# What is (9 + 8) * 2?
return
docs
[
"context"
]
return
docs
[
"context"
]
def
style_01
(
docs
):
def
style_01
(
docs
):
# What is (9 + 8) * 2?
# What is (9 + 8) * 2?
return
docs
[
"context"
].
replace
(
"Question: "
,
""
).
replace
(
" Answer:"
,
""
)
return
docs
[
"context"
].
replace
(
"Question: "
,
""
).
replace
(
" Answer:"
,
""
)
def
style_02
(
docs
):
def
style_02
(
docs
):
# Q: What is (9 + 8) * 2? A:
# Q: What is (9 + 8) * 2? A:
return
docs
[
"context"
].
replace
(
"Question: "
,
"Q: "
).
replace
(
" Answer:"
,
" A:"
)
return
docs
[
"context"
].
replace
(
"Question: "
,
"Q: "
).
replace
(
" Answer:"
,
" A:"
)
def
style_03
(
docs
):
def
style_03
(
docs
):
# Solve (9 + 8) * 2.
# Solve (9 + 8) * 2.
return
docs
[
"context"
].
replace
(
"Question: What is"
,
"Solve"
).
replace
(
" Answer:"
,
"."
)
return
(
docs
[
"context"
].
replace
(
"Question: What is"
,
"Solve"
).
replace
(
" Answer:"
,
"."
)
)
def
style_04
(
docs
):
def
style_04
(
docs
):
# (9 + 8) * 2 =
# (9 + 8) * 2 =
return
docs
[
"context"
].
replace
(
"Question: What is "
,
""
).
replace
(
" Answer:"
,
" ="
)
return
docs
[
"context"
].
replace
(
"Question: What is "
,
""
).
replace
(
" Answer:"
,
" ="
)
def
style_05
(
docs
):
def
style_05
(
docs
):
# What is (9 + 8) * 2? Answer:
# What is (9 + 8) * 2? Answer:
...
...
lm_eval/tasks/hellaswag/alternative_worlds/README.md
View file @
c74e2761
lm_eval/tasks/hellaswag/alternative_worlds/style_01/a.yaml
View file @
c74e2761
lm_eval/tasks/hellaswag/alternative_worlds/style_01/b.yaml
View file @
c74e2761
Prev
1
2
3
4
5
6
7
8
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment