"examples/vscode:/vscode.git/clone" did not exist on "59a85c366ef3666d22b57f952979a3f74ee50f61"
Unverified Commit 23472ff5 authored by Roger Wang's avatar Roger Wang Committed by GitHub
Browse files

[Doc] Add usage of implicit text-only mode (#22561)


Signed-off-by: default avatarRoger Wang <hey@rogerw.me>
Co-authored-by: default avatarFlora Feng <4florafeng@gmail.com>
parent 08b751ba
...@@ -583,6 +583,9 @@ See [this page](../features/multimodal_inputs.md) on how to pass multi-modal inp ...@@ -583,6 +583,9 @@ See [this page](../features/multimodal_inputs.md) on how to pass multi-modal inp
**This is no longer required if you are using vLLM V1.** **This is no longer required if you are using vLLM V1.**
!!! tip
For hybrid-only models such as Llama-4, Step3 and Mistral-3, a text-only mode can be enabled by setting all supported multimodal modalities to 0 (e.g, `--limit-mm-per-prompt '{"image":0}`) so that their multimodal modules will not be loaded to free up more GPU memory for KV cache.
!!! note !!! note
vLLM currently only supports adding LoRA to the language backbone of multimodal models. vLLM currently only supports adding LoRA to the language backbone of multimodal models.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment