Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
dynamo
Commits
120ae649
"vscode:/vscode.git/clone" did not exist on "1e57b1ee6312325a9dab99918422693c38f2b203"
Unverified
Commit
120ae649
authored
Feb 10, 2026
by
Indrajit Bhosale
Committed by
GitHub
Feb 10, 2026
Browse files
fix: llama4 vllm agg multimodal script (#6103)
parent
df8fd92b
Changes
4
Show whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
4 additions
and
23 deletions
+4
-23
docs/features/multimodal/multimodal_vllm.md
docs/features/multimodal/multimodal_vllm.md
+1
-1
examples/backends/vllm/launch/agg_multimodal.sh
examples/backends/vllm/launch/agg_multimodal.sh
+2
-0
examples/backends/vllm/launch/agg_multimodal_llama.sh
examples/backends/vllm/launch/agg_multimodal_llama.sh
+0
-21
fern/pages/features/multimodal/multimodal-vllm.md
fern/pages/features/multimodal/multimodal-vllm.md
+1
-1
No files found.
docs/features/multimodal/multimodal_vllm.md
View file @
120ae649
...
...
@@ -224,7 +224,7 @@ flowchart LR
```
bash
cd
$DYNAMO_HOME
/examples/backends/vllm
bash launch/agg_multimodal
_llama.sh
bash launch/agg_multimodal
.sh
--model
meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8
```
**Client:**
...
...
examples/backends/vllm/launch/agg_multimodal.sh
View file @
120ae649
...
...
@@ -58,6 +58,8 @@ if [[ "$MODEL_NAME" == "Qwen/Qwen2.5-VL-7B-Instruct" ]]; then
MODEL_SPECIFIC_ARGS
=
"--gpu-memory-utilization 0.85 --max-model-len 4096"
elif
[[
"
$MODEL_NAME
"
==
"llava-hf/llava-1.5-7b-hf"
]]
;
then
MODEL_SPECIFIC_ARGS
=
"--gpu-memory-utilization 0.85 --max-model-len 4096"
elif
[[
"
$MODEL_NAME
"
==
"meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8"
]]
;
then
MODEL_SPECIFIC_ARGS
=
"--tensor-parallel-size=8 --gpu-memory-utilization 0.85 --max-model-len=108960"
fi
# Start vLLM worker with vision model
...
...
examples/backends/vllm/launch/agg_multimodal_llama.sh
deleted
100755 → 0
View file @
df8fd92b
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
set
-ex
trap
'echo Cleaning up...; kill 0'
EXIT
MODEL_NAME
=
"meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8"
# run ingress
# dynamo.frontend accepts either --http-port flag or DYN_HTTP_PORT env var (defaults to 8000)
python
-m
dynamo.frontend &
# run processor
python
-m
dynamo.vllm
--multimodal-processor
--enable-multimodal
--model
$MODEL_NAME
&
# Llama 4 doesn't support image embedding input, so use encode+prefill worker
# that handles image encoding inline
python
-m
dynamo.vllm
--multimodal-encode-prefill-worker
--enable-multimodal
--model
$MODEL_NAME
--tensor-parallel-size
=
8
--max-model-len
=
208960
--gpu-memory-utilization
0.80 &
# Wait for all background processes to complete
wait
fern/pages/features/multimodal/multimodal-vllm.md
View file @
120ae649
...
...
@@ -212,7 +212,7 @@ flowchart LR
```
bash
cd
$DYNAMO_HOME
/examples/backends/vllm
bash launch/agg_multimodal
_llama.sh
bash launch/agg_multimodal
.sh
--model
meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8
```
**Client:**
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment