Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
5e11b403
Unverified
Commit
5e11b403
authored
Apr 24, 2026
by
Alex Brooks
Committed by
GitHub
Apr 24, 2026
Browse files
[Frontend] Delegate to vLLM Omni When `--omni` Passed (#40744)
Signed-off-by:
Alex Brooks
<
albrooks@redhat.com
>
parent
f768b447
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
54 additions
and
37 deletions
+54
-37
vllm/entrypoints/cli/main.py
vllm/entrypoints/cli/main.py
+54
-37
No files found.
vllm/entrypoints/cli/main.py
View file @
5e11b403
...
...
@@ -7,6 +7,7 @@ to avoid certain eager import breakage."""
import
importlib.metadata
import
sys
from
importlib.util
import
find_spec
from
vllm.logger
import
init_logger
...
...
@@ -34,6 +35,22 @@ def main():
cli_env_setup
()
# If `--omni` arg is passed to the CLI, delegate to vLLM Omni's entrypoint handling
if
"--omni"
in
sys
.
argv
:
# NOTE: Check the spec instead of importing directly here, since things could
# fail with ImportError due to mismatched versions if things are moved around.
spec
=
find_spec
(
"vllm_omni"
)
if
spec
is
None
:
logger
.
error
(
"--omni flag requires a valid instance of vllm-omni to be installed."
)
sys
.
exit
(
1
)
from
vllm_omni.entrypoints.cli.main
import
main
as
omni_main
logger
.
info
(
"Delegating entrypoint handling to vllm-omni"
)
omni_main
()
else
:
# For 'vllm bench *': use CPU instead of UnspecifiedPlatform by default
if
len
(
sys
.
argv
)
>
1
and
sys
.
argv
[
1
]
==
"bench"
:
logger
.
debug
(
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment