Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
dynamo
Commits
02e59bba
"examples/backends/trtllm/vscode:/vscode.git/clone" did not exist on "17139df9c513c29e1d3bd8fa9106c2b3879b2b2d"
Unverified
Commit
02e59bba
authored
Aug 22, 2025
by
Biswa Panda
Committed by
GitHub
Aug 22, 2025
Browse files
feat: add multimodal deployment example for llava based on vllm v1 (#2628)
parent
e2e909f3
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
68 additions
and
0 deletions
+68
-0
examples/multimodal/deploy/agg_llava.yaml
examples/multimodal/deploy/agg_llava.yaml
+68
-0
No files found.
examples/multimodal/deploy/agg_llava.yaml
0 → 100644
View file @
02e59bba
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
apiVersion
:
nvidia.com/v1alpha1
kind
:
DynamoGraphDeployment
metadata
:
name
:
agg-llava
spec
:
backendFramework
:
vllm
services
:
Frontend
:
dynamoNamespace
:
agg-llava
componentType
:
frontend
replicas
:
1
extraPodSpec
:
mainContainer
:
image
:
my-registry/vllm-runtime:my-tag
EncodeWorker
:
envFromSecret
:
hf-token-secret
dynamoNamespace
:
agg-llava
componentType
:
worker
replicas
:
1
resources
:
limits
:
gpu
:
"
1"
extraPodSpec
:
mainContainer
:
image
:
my-registry/vllm-runtime:my-tag
workingDir
:
/workspace/examples/multimodal
command
:
-
/bin/sh
-
-c
args
:
-
python3 components/encode_worker.py --model llava-hf/llava-1.5-7b-hf
VLMWorker
:
envFromSecret
:
hf-token-secret
dynamoNamespace
:
agg-llava
componentType
:
worker
replicas
:
1
resources
:
limits
:
gpu
:
"
1"
extraPodSpec
:
mainContainer
:
image
:
my-registry/vllm-runtime:my-tag
workingDir
:
/workspace/examples/multimodal
command
:
-
/bin/sh
-
-c
args
:
-
python3 components/worker.py --model llava-hf/llava-1.5-7b-hf --worker-type prefill
Processor
:
envFromSecret
:
hf-token-secret
dynamoNamespace
:
agg-llava
componentType
:
worker
replicas
:
1
resources
:
limits
:
gpu
:
"
1"
extraPodSpec
:
mainContainer
:
image
:
my-registry/vllm-runtime:my-tag
workingDir
:
/workspace/examples/multimodal
command
:
-
/bin/sh
-
-c
args
:
-
'
python3
components/processor.py
--model
llava-hf/llava-1.5-7b-hf
--prompt-template
"USER:
<image>\n<prompt>
ASSISTANT:"'
\ No newline at end of file
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment