Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
dynamo
Commits
120ae649
Unverified
Commit
120ae649
authored
Feb 10, 2026
by
Indrajit Bhosale
Committed by
GitHub
Feb 10, 2026
Browse files
fix: llama4 vllm agg multimodal script (#6103)
parent
df8fd92b
Changes
4
Hide whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
4 additions
and
23 deletions
+4
-23
docs/features/multimodal/multimodal_vllm.md
docs/features/multimodal/multimodal_vllm.md
+1
-1
examples/backends/vllm/launch/agg_multimodal.sh
examples/backends/vllm/launch/agg_multimodal.sh
+2
-0
examples/backends/vllm/launch/agg_multimodal_llama.sh
examples/backends/vllm/launch/agg_multimodal_llama.sh
+0
-21
fern/pages/features/multimodal/multimodal-vllm.md
fern/pages/features/multimodal/multimodal-vllm.md
+1
-1
No files found.
docs/features/multimodal/multimodal_vllm.md
View file @
120ae649
...
@@ -224,7 +224,7 @@ flowchart LR
...
@@ -224,7 +224,7 @@ flowchart LR
```
bash
```
bash
cd
$DYNAMO_HOME
/examples/backends/vllm
cd
$DYNAMO_HOME
/examples/backends/vllm
bash launch/agg_multimodal
_llama.sh
bash launch/agg_multimodal
.sh
--model
meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8
```
```
**Client:**
**Client:**
...
...
examples/backends/vllm/launch/agg_multimodal.sh
View file @
120ae649
...
@@ -58,6 +58,8 @@ if [[ "$MODEL_NAME" == "Qwen/Qwen2.5-VL-7B-Instruct" ]]; then
...
@@ -58,6 +58,8 @@ if [[ "$MODEL_NAME" == "Qwen/Qwen2.5-VL-7B-Instruct" ]]; then
MODEL_SPECIFIC_ARGS
=
"--gpu-memory-utilization 0.85 --max-model-len 4096"
MODEL_SPECIFIC_ARGS
=
"--gpu-memory-utilization 0.85 --max-model-len 4096"
elif
[[
"
$MODEL_NAME
"
==
"llava-hf/llava-1.5-7b-hf"
]]
;
then
elif
[[
"
$MODEL_NAME
"
==
"llava-hf/llava-1.5-7b-hf"
]]
;
then
MODEL_SPECIFIC_ARGS
=
"--gpu-memory-utilization 0.85 --max-model-len 4096"
MODEL_SPECIFIC_ARGS
=
"--gpu-memory-utilization 0.85 --max-model-len 4096"
elif
[[
"
$MODEL_NAME
"
==
"meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8"
]]
;
then
MODEL_SPECIFIC_ARGS
=
"--tensor-parallel-size=8 --gpu-memory-utilization 0.85 --max-model-len=108960"
fi
fi
# Start vLLM worker with vision model
# Start vLLM worker with vision model
...
...
examples/backends/vllm/launch/agg_multimodal_llama.sh
deleted
100755 → 0
View file @
df8fd92b
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
set
-ex
trap
'echo Cleaning up...; kill 0'
EXIT
MODEL_NAME
=
"meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8"
# run ingress
# dynamo.frontend accepts either --http-port flag or DYN_HTTP_PORT env var (defaults to 8000)
python
-m
dynamo.frontend &
# run processor
python
-m
dynamo.vllm
--multimodal-processor
--enable-multimodal
--model
$MODEL_NAME
&
# Llama 4 doesn't support image embedding input, so use encode+prefill worker
# that handles image encoding inline
python
-m
dynamo.vllm
--multimodal-encode-prefill-worker
--enable-multimodal
--model
$MODEL_NAME
--tensor-parallel-size
=
8
--max-model-len
=
208960
--gpu-memory-utilization
0.80 &
# Wait for all background processes to complete
wait
fern/pages/features/multimodal/multimodal-vllm.md
View file @
120ae649
...
@@ -212,7 +212,7 @@ flowchart LR
...
@@ -212,7 +212,7 @@ flowchart LR
```
bash
```
bash
cd
$DYNAMO_HOME
/examples/backends/vllm
cd
$DYNAMO_HOME
/examples/backends/vllm
bash launch/agg_multimodal
_llama.sh
bash launch/agg_multimodal
.sh
--model
meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8
```
```
**Client:**
**Client:**
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment