Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
opencompass
Commits
fed7d800
Unverified
Commit
fed7d800
authored
Feb 04, 2024
by
Yuchen Yan
Committed by
GitHub
Feb 04, 2024
Browse files
[Fix] Fix error in gsm8k evaluator (#782)
Co-authored-by:
jiangjin1999
<
1261842974@qq.com
>
parent
7806cd0f
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
9 additions
and
1 deletion
+9
-1
opencompass/datasets/gsm8k.py
opencompass/datasets/gsm8k.py
+9
-1
No files found.
opencompass/datasets/gsm8k.py
View file @
fed7d800
...
@@ -56,6 +56,14 @@ def gsm8k_postprocess(text: str) -> str:
...
@@ -56,6 +56,14 @@ def gsm8k_postprocess(text: str) -> str:
class
Gsm8kEvaluator
(
BaseEvaluator
):
class
Gsm8kEvaluator
(
BaseEvaluator
):
def
is_equal
(
self
,
pred
,
refer
):
try
:
if
pred
==
refer
or
abs
(
float
(
pred
)
-
int
(
refer
))
<
1e-6
:
return
True
except
Exception
:
pass
return
False
def
score
(
self
,
predictions
,
references
):
def
score
(
self
,
predictions
,
references
):
if
len
(
predictions
)
!=
len
(
references
):
if
len
(
predictions
)
!=
len
(
references
):
return
{
return
{
...
@@ -68,7 +76,7 @@ class Gsm8kEvaluator(BaseEvaluator):
...
@@ -68,7 +76,7 @@ class Gsm8kEvaluator(BaseEvaluator):
for
i
,
j
in
zip
(
predictions
,
references
):
for
i
,
j
in
zip
(
predictions
,
references
):
detail
=
{
'pred'
:
i
,
'answer'
:
j
,
'correct'
:
False
}
detail
=
{
'pred'
:
i
,
'answer'
:
j
,
'correct'
:
False
}
count
+=
1
count
+=
1
if
i
==
j
:
if
self
.
is_equal
(
i
,
j
)
:
correct
+=
1
correct
+=
1
detail
[
'correct'
]
=
True
detail
[
'correct'
]
=
True
details
.
append
(
detail
)
details
.
append
(
detail
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment