Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
dcd44d22
Commit
dcd44d22
authored
Feb 07, 2021
by
Leo Gao
Browse files
LAMBADA metric consistency change
parent
442f843c
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
6 additions
and
6 deletions
+6
-6
lm_eval/tasks/lambada.py
lm_eval/tasks/lambada.py
+6
-6
No files found.
lm_eval/tasks/lambada.py
View file @
dcd44d22
...
...
@@ -53,18 +53,18 @@ class LAMBADA(Task):
ll
,
is_greedy
=
results
return
{
'p
erplexity
'
:
ll
,
'acc
uracy
'
:
int
(
is_greedy
)
'p
pl
'
:
ll
,
'acc'
:
int
(
is_greedy
)
}
def
aggregation
(
self
):
return
{
'p
erplexity
'
:
perplexity
,
'acc
uracy
'
:
mean
'p
pl
'
:
perplexity
,
'acc'
:
mean
}
def
higher_is_better
(
self
):
return
{
'p
erplexity
'
:
False
,
'acc
uracy
'
:
True
'p
pl
'
:
False
,
'acc'
:
True
}
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment