Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
ce1389c9
Commit
ce1389c9
authored
Aug 09, 2023
by
lintangsutawika
Browse files
doc_to_target set to just label
parent
dd8bbe2c
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
1 addition
and
7 deletions
+1
-7
lm_eval/tasks/xcopa/default_et.yaml
lm_eval/tasks/xcopa/default_et.yaml
+1
-1
lm_eval/tasks/xcopa/utils.py
lm_eval/tasks/xcopa/utils.py
+0
-6
No files found.
lm_eval/tasks/xcopa/default_et.yaml
View file @
ce1389c9
...
@@ -6,7 +6,7 @@ output_type: multiple_choice
...
@@ -6,7 +6,7 @@ output_type: multiple_choice
validation_split
:
validation
validation_split
:
validation
test_split
:
test
test_split
:
test
doc_to_text
:
!function
utils.doc_to_text_et
doc_to_text
:
!function
utils.doc_to_text_et
doc_to_target
:
!function
utils.doc_to_target
doc_to_target
:
label
doc_to_choice
:
!function
utils.doc_to_choice
doc_to_choice
:
!function
utils.doc_to_choice
metric_list
:
metric_list
:
-
metric
:
acc
-
metric
:
acc
lm_eval/tasks/xcopa/utils.py
View file @
ce1389c9
...
@@ -11,12 +11,6 @@ def doc_to_text(doc, connector):
...
@@ -11,12 +11,6 @@ def doc_to_text(doc, connector):
return
doc
[
"premise"
].
strip
()[:
-
1
]
+
f
"
{
conn
}
"
return
doc
[
"premise"
].
strip
()[:
-
1
]
+
f
"
{
conn
}
"
def
doc_to_target
(
doc
):
correct_choice
=
doc
[
"choice1"
]
if
doc
[
"label"
]
==
0
else
doc
[
"choice2"
]
# Connect the sentences
return
convert_choice
(
correct_choice
)
def
doc_to_choice
(
doc
):
def
doc_to_choice
(
doc
):
return
[
convert_choice
(
doc
[
"choice1"
]),
convert_choice
(
doc
[
"choice2"
])]
return
[
convert_choice
(
doc
[
"choice1"
]),
convert_choice
(
doc
[
"choice2"
])]
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment