Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
34c60176
Commit
34c60176
authored
Mar 13, 2023
by
ingyuseong
Browse files
Refactor KLUE-STS calculating pred
parent
26e55d33
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
4 deletions
+3
-4
lm_eval/tasks/klue.py
lm_eval/tasks/klue.py
+3
-4
No files found.
lm_eval/tasks/klue.py
View file @
34c60176
...
...
@@ -61,13 +61,12 @@ class STS(Task):
return
" {}"
.
format
({
0
:
"아니오"
,
1
:
"예"
}[
doc
[
"labels"
][
"binary-label"
]])
def
construct_requests
(
self
,
doc
,
ctx
):
ll_positive
,
_
=
rf
.
loglikelihood
(
ctx
,
" 예"
)
ll_negative
,
_
=
rf
.
loglikelihood
(
ctx
,
" 아니오"
)
return
ll_positive
,
ll_negative
ll_positive
,
_
=
rf
.
loglikelihood
(
ctx
,
" 예"
)
return
ll_negative
,
ll_positive
def
process_results
(
self
,
doc
,
results
):
ll_positive
,
ll_negative
=
results
pred
=
ll_positive
>
ll_negative
pred
=
np
.
argmax
(
results
)
gold
=
doc
[
"labels"
][
"binary-label"
]
return
{
"acc"
:
pred
==
gold
,
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment