Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
dd5b526b
Commit
dd5b526b
authored
Aug 28, 2023
by
lintangsutawika
Browse files
format
parent
04f5697d
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
4 deletions
+4
-4
lm_eval/tasks/wsc273/utils.py
lm_eval/tasks/wsc273/utils.py
+4
-4
No files found.
lm_eval/tasks/wsc273/utils.py
View file @
dd5b526b
upper_pronouns
=
[
"A"
,
"An"
,
...
...
@@ -13,17 +12,18 @@ upper_pronouns = [
"Their"
,
]
def
process_doc
(
dataset
):
def
process_doc
(
dataset
):
def
process_fn
(
doc
):
# The HF implementation of `wsc273` is not `partial evaluation` friendly.
doc
[
"text"
]
=
doc
[
"text"
].
replace
(
" "
,
" "
)
doc
[
"options"
][
0
]
=
__normalize_option
(
doc
,
doc
[
"options"
][
0
])
doc
[
"options"
][
1
]
=
__normalize_option
(
doc
,
doc
[
"options"
][
1
])
return
doc
return
dataset
.
map
(
process_fn
)
def
__normalize_option
(
doc
,
option
):
# Append `'s` to possessive determiner based options.
if
doc
[
"pronoun"
].
lower
()
in
[
"my"
,
"his"
,
"her"
,
"our"
,
"their"
]:
...
...
@@ -33,4 +33,4 @@ def __normalize_option(doc, option):
start_of_sentence
=
doc
[
"text"
][
doc
[
"pronoun_loc"
]
-
2
]
==
"."
if
not
start_of_sentence
and
pronoun
in
upper_pronouns
:
return
option
.
replace
(
pronoun
,
pronoun
.
lower
())
return
option
\ No newline at end of file
return
option
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment