Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
eae3d481
Unverified
Commit
eae3d481
authored
Oct 29, 2024
by
Cyrus Leung
Committed by
GitHub
Oct 28, 2024
Browse files
[Bugfix] Use temporary directory in registry (#9721)
parent
e74f2d44
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
8 additions
and
3 deletions
+8
-3
vllm/model_executor/models/registry.py
vllm/model_executor/models/registry.py
+8
-3
No files found.
vllm/model_executor/models/registry.py
View file @
eae3d481
import
importlib
import
importlib
import
os
import
pickle
import
pickle
import
subprocess
import
subprocess
import
sys
import
sys
...
@@ -423,9 +424,13 @@ _T = TypeVar("_T")
...
@@ -423,9 +424,13 @@ _T = TypeVar("_T")
def
_run_in_subprocess
(
fn
:
Callable
[[],
_T
])
->
_T
:
def
_run_in_subprocess
(
fn
:
Callable
[[],
_T
])
->
_T
:
with
tempfile
.
NamedTemporaryFile
()
as
output_file
:
# NOTE: We use a temporary directory instead of a temporary file to avoid
# issues like https://stackoverflow.com/questions/23212435/permission-denied-to-write-to-my-temporary-file
with
tempfile
.
TemporaryDirectory
()
as
tempdir
:
output_filepath
=
os
.
path
.
join
(
tempdir
,
"registry_output.tmp"
)
# `cloudpickle` allows pickling lambda functions directly
# `cloudpickle` allows pickling lambda functions directly
input_bytes
=
cloudpickle
.
dumps
((
fn
,
output_file
.
name
))
input_bytes
=
cloudpickle
.
dumps
((
fn
,
output_file
path
))
# cannot use `sys.executable __file__` here because the script
# cannot use `sys.executable __file__` here because the script
# contains relative imports
# contains relative imports
...
@@ -442,7 +447,7 @@ def _run_in_subprocess(fn: Callable[[], _T]) -> _T:
...
@@ -442,7 +447,7 @@ def _run_in_subprocess(fn: Callable[[], _T]) -> _T:
raise
RuntimeError
(
f
"Error raised in subprocess:
\n
"
raise
RuntimeError
(
f
"Error raised in subprocess:
\n
"
f
"
{
returned
.
stderr
.
decode
()
}
"
)
from
e
f
"
{
returned
.
stderr
.
decode
()
}
"
)
from
e
with
open
(
output_file
.
name
,
"rb"
)
as
f
:
with
open
(
output_file
path
,
"rb"
)
as
f
:
return
pickle
.
load
(
f
)
return
pickle
.
load
(
f
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment