Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
opencompass
Commits
80f831b4
Unverified
Commit
80f831b4
authored
May 15, 2024
by
Fengzhe Zhou
Committed by
GitHub
May 15, 2024
Browse files
[Fix] use ProcessPoolExecutor during mbpp eval (#1159)
parent
8a8987be
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
4 deletions
+3
-4
opencompass/datasets/mbpp.py
opencompass/datasets/mbpp.py
+3
-4
No files found.
opencompass/datasets/mbpp.py
View file @
80f831b4
...
...
@@ -8,7 +8,7 @@ import re
import
signal
import
tempfile
from
collections
import
defaultdict
from
concurrent.futures
import
Thread
PoolExecutor
,
as_completed
from
concurrent.futures
import
Process
PoolExecutor
,
as_completed
from
typing
import
List
,
Sequence
,
Union
import
numpy
as
np
...
...
@@ -213,8 +213,7 @@ class MBPPEvaluator(BaseEvaluator):
if
self
.
metric
==
'MBPP'
:
result
=
{
'pass'
:
0
,
'timeout'
:
0
,
'failed'
:
0
,
'wrong_answer'
:
0
}
details
=
{}
# change to thread pool for better killing blocked instance
with
ThreadPoolExecutor
()
as
executor
:
with
ProcessPoolExecutor
()
as
executor
:
futures
=
[]
for
i
,
(
refer
,
pred
)
in
enumerate
(
zip
(
references
,
predictions
)):
...
...
@@ -439,7 +438,7 @@ class MBPPPassKEvaluator(MBPPEvaluator):
task_total
=
defaultdict
(
int
)
result
=
{
'pass'
:
0
,
'timeout'
:
0
,
'failed'
:
0
,
'wrong_answer'
:
0
}
with
Thread
PoolExecutor
()
as
executor
:
with
Process
PoolExecutor
()
as
executor
:
futures
=
[]
for
refer
,
preds
in
zip
(
references
,
predictions
):
# suits for two case
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment