Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
6724e791
Unverified
Commit
6724e791
authored
Feb 25, 2025
by
Cyrus Leung
Committed by
GitHub
Feb 25, 2025
Browse files
[Misc] Check that the model can be inspected upon registration (#13743)
parent
03f48b3d
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
15 additions
and
1 deletion
+15
-1
vllm/model_executor/models/registry.py
vllm/model_executor/models/registry.py
+15
-1
No files found.
vllm/model_executor/models/registry.py
View file @
6724e791
...
...
@@ -347,6 +347,10 @@ class _ModelRegistry:
when importing the model and thus the related error
:code:`RuntimeError: Cannot re-initialize CUDA in forked subprocess`.
"""
if
not
isinstance
(
model_arch
,
str
):
msg
=
f
"`model_arch` should be a string, not a
{
type
(
model_arch
)
}
"
raise
TypeError
(
msg
)
if
model_arch
in
self
.
models
:
logger
.
warning
(
"Model architecture %s is already registered, and will be "
...
...
@@ -360,8 +364,18 @@ class _ModelRegistry:
raise
ValueError
(
msg
)
model
=
_LazyRegisteredModel
(
*
split_str
)
else
:
try
:
model
.
inspect_model_cls
()
except
Exception
as
exc
:
msg
=
f
"Unable to inspect model
{
model_cls
}
"
raise
RuntimeError
(
msg
)
from
exc
elif
isinstance
(
model_cls
,
type
)
and
issubclass
(
model_cls
,
nn
.
Module
):
model
=
_RegisteredModel
.
from_model_cls
(
model_cls
)
else
:
msg
=
(
"`model_cls` should be a string or PyTorch model class, "
f
"not a
{
type
(
model_arch
)
}
"
)
raise
TypeError
(
msg
)
self
.
models
[
model_arch
]
=
model
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment