Unverified Commit 8291bfcc authored by Ayush Agarwal's avatar Ayush Agarwal Committed by GitHub
Browse files

docs(vllm-omni): note ARM64 not supported; flag GLM-Image as experimental (#8373)


Signed-off-by: default avatarayushag <ayushag@nvidia.com>
parent 073da7b1
......@@ -12,12 +12,14 @@ This guide assumes familiarity with deploying Dynamo with vLLM as described in t
### Installation
Dynamo container images include vLLM-Omni pre-installed. If you are using `pip install ai-dynamo[vllm]`, vLLM-Omni is **not** included automatically because the matching release is not yet available on PyPI. Install it separately from source:
Dynamo container images include vLLM-Omni pre-installed. If you are using `pip install ai-dynamo[vllm]`, vLLM-Omni is **not** included automatically because the matching release is not yet available on PyPI. Install it separately from source, pinning the vLLM-Omni release that matches your installed vLLM version (see the [vLLM-Omni releases](https://github.com/vllm-project/vllm-omni/releases) page):
```bash
pip install git+https://github.com/vllm-project/vllm-omni.git@v0.16.0rc1
pip install git+https://github.com/vllm-project/vllm-omni.git@<version>
```
> **ARM64 not supported:** vLLM-Omni is currently only installed on `amd64` builds. On `arm64`, the container build skips the install and vLLM-Omni features are unavailable.
## Supported Modalities
| Modality | Endpoint(s) | `--output-modalities` |
......@@ -372,6 +374,8 @@ sequenceDiagram
GLM-Image is a 2-stage text-to-image model with an AR stage (generates prior token IDs) and a DiT stage (diffusion denoising + VAE decode). The built-in vLLM-Omni stage config already assigns each stage to a separate GPU.
> **Experimental:** GLM-Image support is experimental; generation may fail or produce incorrect/garbled outputs for some prompts and sizes.
>
> **Known issue:** GLM-Image requires `transformers>=5.0` to recognize the `glm_image` architecture. Older versions fail at model config creation with `The checkpoint you are trying to load has model type 'glm_image' but Transformers does not recognize this architecture`.
```bash
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment