Skip to content

GitLab

"container/vscode:/vscode.git/clone" did not exist on "f80ad6926a0f403c8e9486203dadc55e767f29b1"
Unverified Commit 24fe435d authored by dagil-nvidia's avatar dagil-nvidia Committed by GitHub
Browse files

docs: update support matrix, release artifacts, and README for v1.0.0 (#7350) 


Signed-off-by: default avatarDan Gil <dagil@nvidia.com>
parent e6cb6847
Show whitespace changes
Inline Side-by-side
Showing with 125 additions and 66 deletions
README.md
View file @ 24fe435d
......@@ -97,13 +97,13 @@ Containers have all dependencies pre-installed. No setup required.
```bash
# SGLang
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:0.8.1
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.0.0
# TensorRT-LLM
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:0.8.1
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.0.0
# vLLM
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/vllm-runtime:0.8.1
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.0.0
```
> **Tip:** To run frontend and worker in the same container, either run processes in background with `&` (see below), or open a second terminal and use `docker exec -it <container_id> bash`.
......
docs/reference/release-artifacts.md
View file @ 24fe435d
......@@ -10,34 +10,26 @@ This document provides a comprehensive inventory of all Dynamo release artifacts
Release history in this document begins at v0.6.0.
## Current Release: Dynamo v0.9.1
## Current Release: Dynamo v1.0.0
- **GitHub Release:** [v0.9.1](https://github.com/ai-dynamo/dynamo/releases/tag/v0.9.1)
- **Docs:** [v0.9.1](https://docs.dynamo.nvidia.com/dynamo)
- **GitHub Release:** [v1.0.0](https://github.com/ai-dynamo/dynamo/releases/tag/v1.0.0)
- **Docs:** [v1.0.0](https://docs.dynamo.nvidia.com/dynamo)
- **NGC Collection:** [ai-dynamo](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/collections/ai-dynamo)
### Previous Patch: v0.9.0.post1 (Feb 2026)
**v0.9.0.post1** is a Helm-chart-only patch release on NGC (no GitHub release). It fixes the `dynamo-platform` Helm chart which incorrectly set the operator image tag to `0.7.1` instead of `0.9.0`. Only the `dynamo-platform` chart was patched; all other artifacts remain at v0.9.0. Users upgrading to v0.9.1 do not need this patch.
| Artifact | Version | Change | Link |
|----------|---------|--------|------|
| `dynamo-platform` | `0.9.0-post1` | Fixed operator image tag (`0.7.1` -> `0.9.0`) | [NGC](https://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-platform-0.9.0-post1.tgz) |
**Workaround for v0.9.0 chart:** If using the original v0.9.0 Helm chart, add this flag:
`--set dynamo-operator.controllerManager.manager.image.tag=0.9.0`
### Container Images
| Image:Tag | Description | Backend | CUDA | Arch | NGC | Notes |
|-----------|-------------|---------|------|------|-----|-------|
| `vllm-runtime:0.9.1` | Runtime container for vLLM backend | vLLM `v0.14.1` | `v12.9` | AMD64/ARM64 | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/vllm-runtime?version=0.9.1) | |
| `vllm-runtime:0.9.1-cuda13` | Runtime container for vLLM backend (CUDA 13) | vLLM `v0.14.1` | `v13.0` | AMD64/ARM64* | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/vllm-runtime?version=0.9.1-cuda13) | Experimental |
| `sglang-runtime:0.9.1` | Runtime container for SGLang backend | SGLang `v0.5.8` | `v12.9` | AMD64/ARM64 | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/sglang-runtime?version=0.9.1) | |
| `sglang-runtime:0.9.1-cuda13` | Runtime container for SGLang backend (CUDA 13) | SGLang `v0.5.8` | `v13.0` | AMD64/ARM64* | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/sglang-runtime?version=0.9.1-cuda13) | Experimental |
| `tensorrtllm-runtime:0.9.1` | Runtime container for TensorRT-LLM backend | TRT-LLM `v1.3.0rc3` | `v13.0` | AMD64/ARM64 | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/tensorrtllm-runtime?version=0.9.1) | |
| `dynamo-frontend:0.9.1` | API gateway with Endpoint Prediction Protocol (EPP) | — | — | AMD64/ARM64 | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/dynamo-frontend?version=0.9.1) | |
| `kubernetes-operator:0.9.1` | Kubernetes operator for Dynamo deployments | — | — | AMD64/ARM64 | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/kubernetes-operator?version=0.9.1) | |
| `vllm-runtime:1.0.0` | Runtime container for vLLM backend | vLLM `v0.16.0` | `v12.9` | AMD64/ARM64 | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/vllm-runtime?version=1.0.0) | |
| `vllm-runtime:1.0.0-cuda13` | Runtime container for vLLM backend (CUDA 13) | vLLM `v0.16.0` | `v13.0` | AMD64/ARM64* | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/vllm-runtime?version=1.0.0-cuda13) | |
| `vllm-runtime:1.0.0-efa-amd64` | Runtime container for vLLM with AWS EFA | vLLM `v0.16.0` | `v12.9` | AMD64 | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/vllm-runtime?version=1.0.0-efa-amd64) | Experimental |
| `sglang-runtime:1.0.0` | Runtime container for SGLang backend | SGLang `v0.5.9` | `v12.9` | AMD64/ARM64 | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/sglang-runtime?version=1.0.0) | |
| `sglang-runtime:1.0.0-cuda13` | Runtime container for SGLang backend (CUDA 13) | SGLang `v0.5.9` | `v13.0` | AMD64/ARM64* | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/sglang-runtime?version=1.0.0-cuda13) | |
| `tensorrtllm-runtime:1.0.0` | Runtime container for TensorRT-LLM backend | TRT-LLM `v1.3.0rc5.post1` | `v13.1` | AMD64/ARM64 | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/tensorrtllm-runtime?version=1.0.0) | |
| `tensorrtllm-runtime:1.0.0-efa-amd64` | Runtime container for TensorRT-LLM with AWS EFA | TRT-LLM `v1.3.0rc5.post1` | `v13.1` | AMD64 | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/tensorrtllm-runtime?version=1.0.0-efa-amd64) | Experimental |
| `dynamo-frontend:1.0.0` | API gateway with Endpoint Prediction Protocol (EPP) | — | — | AMD64/ARM64 | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/dynamo-frontend?version=1.0.0) | |
| `kubernetes-operator:1.0.0` | Kubernetes operator for Dynamo deployments | — | — | AMD64/ARM64 | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/kubernetes-operator?version=1.0.0) | |
| `snapshot-agent:1.0.0` | Snapshot agent for fast GPU worker recovery via CRIU | — | — | AMD64/ARM64 | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/snapshot-agent?version=1.0.0) | Preview |
\* Multimodal inference on CUDA 13 images: works on AMD64 for all backends; works on ARM64 only for TensorRT-LLM (`vllm-runtime:*-cuda13` and `sglang-runtime:*-cuda13` do not support multimodality on ARM64).
......@@ -47,30 +39,32 @@ We recommend using the TensorRT-LLM NGC container instead of the `ai-dynamo[trtl
| Package | Description | Python | Platform | PyPI |
|---------|-------------|--------|----------|------|
| `ai-dynamo==0.9.1` | Main package with backend integrations (vLLM, SGLang, TRT-LLM) | `3.10`–`3.12` | Linux (glibc `v2.28+`) | [link](https://pypi.org/project/ai-dynamo/0.9.1/) |
| `ai-dynamo-runtime==0.9.1` | Core Python bindings for Dynamo runtime | `3.10`–`3.12` | Linux (glibc `v2.28+`) | [link](https://pypi.org/project/ai-dynamo-runtime/0.9.1/) |
| `kvbm==0.9.1` | KV Block Manager for disaggregated KV cache | `3.12` | Linux (glibc `v2.28+`) | [link](https://pypi.org/project/kvbm/0.9.1/) |
| `ai-dynamo==1.0.0` | Main package with backend integrations (vLLM, SGLang, TRT-LLM) | `3.10`–`3.12` | Linux (glibc `v2.28+`) | [link](https://pypi.org/project/ai-dynamo/1.0.0/) |
| `ai-dynamo-runtime==1.0.0` | Core Python bindings for Dynamo runtime | `3.10`–`3.12` | Linux (glibc `v2.28+`) | [link](https://pypi.org/project/ai-dynamo-runtime/1.0.0/) |
| `kvbm==1.0.0` | KV Block Manager for disaggregated KV cache | `3.12` | Linux (glibc `v2.28+`) | [link](https://pypi.org/project/kvbm/1.0.0/) |
### Helm Charts
| Chart | Description | NGC |
|-------|-------------|-----|
| `dynamo-crds-0.9.1` | Custom Resource Definitions for Dynamo Kubernetes resources | [link](https://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-crds-0.9.1.tgz) |
| `dynamo-platform-0.9.1` | Platform services (etcd, NATS) for Dynamo cluster | [link](https://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-platform-0.9.1.tgz) |
| `dynamo-platform-1.0.0` | Platform services (etcd, NATS) and Dynamo Operator for Dynamo cluster | [link](https://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-platform-1.0.0.tgz) |
| `snapshot-1.0.0` | Snapshot DaemonSet for fast GPU worker recovery | [link](https://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/snapshot-1.0.0.tgz) |
> **Note:** The `dynamo-graph` Helm chart is deprecated as of v0.9.0. Use the Kubernetes operator for deployment graph management.
> **Note:** The `dynamo-crds` Helm chart is deprecated as of v1.0.0; CRDs are now managed by the Dynamo Operator. The `dynamo-graph` Helm chart is deprecated as of v0.9.0.
### Rust Crates
| Crate | Description | MSRV (Rust) | crates.io |
|-------|-------------|-------------|-----------|
| `dynamo-runtime@0.9.1` | Core distributed runtime library | `v1.82` | [link](https://crates.io/crates/dynamo-runtime/0.9.1) |
| `dynamo-llm@0.9.1` | LLM inference engine | `v1.82` | [link](https://crates.io/crates/dynamo-llm/0.9.1) |
| `dynamo-async-openai@0.9.1` | Async OpenAI-compatible API client | `v1.82` | [link](https://crates.io/crates/dynamo-async-openai/0.9.1) |
| `dynamo-parsers@0.9.1` | Protocol parsers (SSE, JSON streaming) | `v1.82` | [link](https://crates.io/crates/dynamo-parsers/0.9.1) |
| `dynamo-memory@0.9.1` | Memory management utilities | `v1.82` | [link](https://crates.io/crates/dynamo-memory/0.9.1) |
| `dynamo-config@0.9.1` | Configuration management | `v1.82` | [link](https://crates.io/crates/dynamo-config/0.9.1) |
| `dynamo-tokens@0.9.1` | Tokenizer bindings for LLM inference | `v1.82` | [link](https://crates.io/crates/dynamo-tokens/0.9.1) |
| `dynamo-runtime@1.0.0` | Core distributed runtime library | `v1.82` | [link](https://crates.io/crates/dynamo-runtime/1.0.0) |
| `dynamo-llm@1.0.0` | LLM inference engine | `v1.82` | [link](https://crates.io/crates/dynamo-llm/1.0.0) |
| `dynamo-async-openai@1.0.0` | Async OpenAI-compatible API client | `v1.82` | [link](https://crates.io/crates/dynamo-async-openai/1.0.0) |
| `dynamo-parsers@1.0.0` | Protocol parsers (SSE, JSON streaming) | `v1.82` | [link](https://crates.io/crates/dynamo-parsers/1.0.0) |
| `dynamo-memory@1.0.0` | Memory management utilities | `v1.82` | [link](https://crates.io/crates/dynamo-memory/1.0.0) |
| `dynamo-config@1.0.0` | Configuration management | `v1.82` | [link](https://crates.io/crates/dynamo-config/1.0.0) |
| `dynamo-tokens@1.0.0` | Tokenizer bindings for LLM inference | `v1.82` | [link](https://crates.io/crates/dynamo-tokens/1.0.0) |
| `dynamo-mocker@1.0.0` | Inference engine simulator for benchmarking | `v1.82` | [link](https://crates.io/crates/dynamo-mocker/1.0.0) |
| `dynamo-kv-router@1.0.0` | KV-aware request routing library | `v1.82` | [link](https://crates.io/crates/dynamo-kv-router/1.0.0) |
## Quick Install Commands
......@@ -80,17 +74,22 @@ We recommend using the TensorRT-LLM NGC container instead of the `ai-dynamo[trtl
```bash
# Runtime containers
docker pull nvcr.io/nvidia/ai-dynamo/vllm-runtime:0.9.1
docker pull nvcr.io/nvidia/ai-dynamo/sglang-runtime:0.9.1
docker pull nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:0.9.1
docker pull nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.0.0
docker pull nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.0.0
docker pull nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.0.0
# CUDA 13 variants
docker pull nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.0.0-cuda13
docker pull nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.0.0-cuda13
# CUDA 13 variants (experimental)
docker pull nvcr.io/nvidia/ai-dynamo/vllm-runtime:0.9.1-cuda13
docker pull nvcr.io/nvidia/ai-dynamo/sglang-runtime:0.9.1-cuda13
# EFA variants (AWS, AMD64 only, experimental)
docker pull nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.0.0-efa-amd64
docker pull nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.0.0-efa-amd64
# Infrastructure containers
docker pull nvcr.io/nvidia/ai-dynamo/dynamo-frontend:0.9.1
docker pull nvcr.io/nvidia/ai-dynamo/kubernetes-operator:0.9.1
docker pull nvcr.io/nvidia/ai-dynamo/dynamo-frontend:1.0.0
docker pull nvcr.io/nvidia/ai-dynamo/kubernetes-operator:1.0.0
docker pull nvcr.io/nvidia/ai-dynamo/snapshot-agent:1.0.0
```
### Python Wheels (PyPI)
......@@ -99,16 +98,16 @@ docker pull nvcr.io/nvidia/ai-dynamo/kubernetes-operator:0.9.1
```bash
# Install Dynamo with a specific backend (Recommended)
uv pip install "ai-dynamo[vllm]==0.9.1"
uv pip install "ai-dynamo[sglang]==0.9.1"
uv pip install "ai-dynamo[vllm]==1.0.0"
uv pip install "ai-dynamo[sglang]==1.0.0"
# TensorRT-LLM requires the NVIDIA PyPI index and pip
pip install --pre --extra-index-url https://pypi.nvidia.com "ai-dynamo[trtllm]==0.9.1"
pip install --pre --extra-index-url https://pypi.nvidia.com "ai-dynamo[trtllm]==1.0.0"
# Install Dynamo core only
uv pip install ai-dynamo==0.9.1
uv pip install ai-dynamo==1.0.0
# Install standalone KVBM (Python 3.12 only)
uv pip install kvbm==0.9.1
uv pip install kvbm==1.0.0
```
### Helm Charts (NGC)
......@@ -116,8 +115,8 @@ uv pip install kvbm==0.9.1
> For Kubernetes deployment instructions, see the [Kubernetes Installation Guide](../kubernetes/installation-guide.md).
```bash
helm install dynamo-crds oci://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-crds --version 0.9.1
helm install dynamo-platform oci://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-platform --version 0.9.1
helm install dynamo-platform oci://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-platform --version 1.0.0
helm install snapshot oci://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/snapshot --version 1.0.0
```
### Rust Crates (crates.io)
......@@ -125,13 +124,15 @@ helm install dynamo-platform oci://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/d
> For API documentation, see each crate on [docs.rs](https://docs.rs/). To build Dynamo from source, see [Building from Source](https://github.com/ai-dynamo/dynamo#building-from-source).
```bash
cargo add dynamo-runtime@0.9.1
cargo add dynamo-llm@0.9.1
cargo add dynamo-async-openai@0.9.1
cargo add dynamo-parsers@0.9.1
cargo add dynamo-memory@0.9.1
cargo add dynamo-config@0.9.1
cargo add dynamo-tokens@0.9.1
cargo add dynamo-runtime@1.0.0
cargo add dynamo-llm@1.0.0
cargo add dynamo-async-openai@1.0.0
cargo add dynamo-parsers@1.0.0
cargo add dynamo-memory@1.0.0
cargo add dynamo-config@1.0.0
cargo add dynamo-tokens@1.0.0
cargo add dynamo-mocker@1.0.0
cargo add dynamo-kv-router@1.0.0
```
**CUDA and Driver Requirements:** For detailed CUDA toolkit versions and minimum driver requirements for each container image, see the [Support Matrix](support-matrix.md#cuda-and-driver-requirements).
......@@ -139,6 +140,7 @@ cargo add dynamo-tokens@0.9.1
## Known Issues
For a complete list of known issues, refer to the release notes for each version:
- [v1.0.0 Release Notes](https://github.com/ai-dynamo/dynamo/releases/tag/v1.0.0)
- [v0.9.0 Release Notes](https://github.com/ai-dynamo/dynamo/releases/tag/v0.9.0)
- [v0.8.1 Release Notes](https://github.com/ai-dynamo/dynamo/releases/tag/v0.8.1)
......@@ -155,11 +157,13 @@ For a complete list of known issues, refer to the release notes for each version
## Release History
- **v1.0.0**: First major release. SGLang `v0.5.9`, TRT-LLM `v1.3.0rc5.post1` (CUDA 13.1), vLLM `v0.16.0`, NIXL `v0.10.1`. New `snapshot-agent` container and `snapshot` Helm chart (Preview). New EFA container variants for vLLM and TRT-LLM (Experimental, AMD64 only). New `dynamo-mocker` and `dynamo-kv-router` Rust crates. Deprecated `dynamo-crds` Helm chart (CRDs now managed by the Operator). `v1alpha1` CRDs deprecated.
- **v0.9.1**: Updated TRT-LLM to `v1.3.0rc3`. All other backend versions unchanged from v0.9.0.
- **v0.9.0.post1**: Fixed `dynamo-platform` Helm chart operator image tag (Helm chart only, NGC)
- **v0.9.0**: Updated vLLM to `v0.14.1`, SGLang to `v0.5.8`, TRT-LLM to `v1.3.0rc1`, NIXL to `v0.9.0`. New `dynamo-tokens` Rust crate. Deprecated `dynamo-graph` Helm chart.
- **v0.8.1.post1/.post2/.post3 Patches**: Experimental patch releases updating TRT-LLM only (PyPI wheels and TRT-LLM container). No other artifacts changed.
- **Standalone Frontend Container**: `dynamo-frontend` added in v0.8.0
- **EFA Runtimes**: Experimental AWS EFA variants for vLLM and TRT-LLM (AMD64 only) in v1.0.0
- **CUDA 13 Runtimes**: Experimental CUDA 13 runtime for SGLang and vLLM in v0.8.0
- **New Rust Crates**: `dynamo-memory` and `dynamo-config` added in v0.8.0
......@@ -167,6 +171,7 @@ For a complete list of known issues, refer to the release notes for each version
| Version | Release Date | GitHub | Docs |
|---------|--------------|--------|------|
| `v1.0.0` | Mar 2026 | [Release](https://github.com/ai-dynamo/dynamo/releases/tag/v1.0.0) | [Docs](https://docs.dynamo.nvidia.com/dynamo) |
| `v0.9.1` | Mar 4, 2026 | [Release](https://github.com/ai-dynamo/dynamo/releases/tag/v0.9.1) | [Docs](https://docs.dynamo.nvidia.com/dynamo) |
| `v0.9.0` | Feb 11, 2026 | [Release](https://github.com/ai-dynamo/dynamo/releases/tag/v0.9.0) | [Docs](https://docs.dynamo.nvidia.com/dynamo/v-0-9-0/) |
| `v0.8.1` | Jan 23, 2026 | [Release](https://github.com/ai-dynamo/dynamo/releases/tag/v0.8.1) | [Docs](https://docs.nvidia.com/dynamo/v-0-8-1/) |
......@@ -187,6 +192,9 @@ For a complete list of known issues, refer to the release notes for each version
| Image:Tag | vLLM | Arch | CUDA | Notes |
|-----------|------|------|------|-------|
| `vllm-runtime:1.0.0` | `v0.16.0` | AMD64/ARM64 | `v12.9` | |
| `vllm-runtime:1.0.0-cuda13` | `v0.16.0` | AMD64/ARM64* | `v13.0` | |
| `vllm-runtime:1.0.0-efa-amd64` | `v0.16.0` | AMD64 | `v12.9` | Experimental |
| `vllm-runtime:0.9.1` | `v0.14.1` | AMD64/ARM64 | `v12.9` | |
| `vllm-runtime:0.9.1-cuda13` | `v0.14.1` | AMD64/ARM64* | `v13.0` | Experimental |
| `vllm-runtime:0.9.0` | `v0.14.1` | AMD64/ARM64 | `v12.9` | |
......@@ -206,6 +214,8 @@ For a complete list of known issues, refer to the release notes for each version
| Image:Tag | SGLang | Arch | CUDA | Notes |
|-----------|--------|------|------|-------|
| `sglang-runtime:1.0.0` | `v0.5.9` | AMD64/ARM64 | `v12.9` | |
| `sglang-runtime:1.0.0-cuda13` | `v0.5.9` | AMD64/ARM64* | `v13.0` | |
| `sglang-runtime:0.9.1` | `v0.5.8` | AMD64/ARM64 | `v12.9` | |
| `sglang-runtime:0.9.1-cuda13` | `v0.5.8` | AMD64/ARM64* | `v13.0` | Experimental |
| `sglang-runtime:0.9.0` | `v0.5.8` | AMD64/ARM64 | `v12.9` | |
......@@ -225,6 +235,8 @@ For a complete list of known issues, refer to the release notes for each version
| Image:Tag | TRT-LLM | Arch | CUDA | Notes |
|-----------|---------|------|------|-------|
| `tensorrtllm-runtime:1.0.0` | `v1.3.0rc5.post1` | AMD64/ARM64 | `v13.1` | |
| `tensorrtllm-runtime:1.0.0-efa-amd64` | `v1.3.0rc5.post1` | AMD64 | `v13.1` | Experimental |
| `tensorrtllm-runtime:0.9.1` | `v1.3.0rc3` | AMD64/ARM64 | `v13.0` | |
| `tensorrtllm-runtime:0.9.0` | `v1.3.0rc1` | AMD64/ARM64 | `v13.0` | |
| `tensorrtllm-runtime:0.8.1.post3` | `v1.2.0rc6.post3` | AMD64/ARM64 | `v13.0` | Patch |
......@@ -244,6 +256,7 @@ For a complete list of known issues, refer to the release notes for each version
| Image:Tag | Arch | Notes |
|-----------|------|-------|
| `dynamo-frontend:1.0.0` | AMD64/ARM64 | |
| `dynamo-frontend:0.9.1` | AMD64/ARM64 | |
| `dynamo-frontend:0.9.0` | AMD64/ARM64 | |
| `dynamo-frontend:0.8.1` | AMD64/ARM64 | |
......@@ -253,6 +266,7 @@ For a complete list of known issues, refer to the release notes for each version
| Image:Tag | Arch | Notes |
|-----------|------|-------|
| `kubernetes-operator:1.0.0` | AMD64/ARM64 | |
| `kubernetes-operator:0.9.1` | AMD64/ARM64 | |
| `kubernetes-operator:0.9.0` | AMD64/ARM64 | |
| `kubernetes-operator:0.8.1` | AMD64/ARM64 | |
......@@ -263,6 +277,12 @@ For a complete list of known issues, refer to the release notes for each version
| `kubernetes-operator:0.6.1` | AMD64/ARM64 | |
| `kubernetes-operator:0.6.0` | AMD64/ARM64 | |
#### snapshot-agent
| Image:Tag | Arch | Notes |
|-----------|------|-------|
| `snapshot-agent:1.0.0` | AMD64/ARM64 | Preview |
### Python Wheels
> **PyPI:** [ai-dynamo](https://pypi.org/project/ai-dynamo/) | [ai-dynamo-runtime](https://pypi.org/project/ai-dynamo-runtime/) | [kvbm](https://pypi.org/project/kvbm/)
......@@ -273,6 +293,7 @@ For a complete list of known issues, refer to the release notes for each version
| Package | Python | Platform | Notes |
|---------|--------|----------|-------|
| `ai-dynamo==1.0.0` | `3.10`–`3.12` | Linux (glibc `v2.28+`) | |
| `ai-dynamo==0.9.1` | `3.10`–`3.12` | Linux (glibc `v2.28+`) | |
| `ai-dynamo==0.9.0` | `3.10`–`3.12` | Linux (glibc `v2.28+`) | |
| `ai-dynamo==0.8.1.post3` | `3.10`–`3.12` | Linux (glibc `v2.28+`) | TRT-LLM `v1.2.0rc6.post3` |
......@@ -288,6 +309,7 @@ For a complete list of known issues, refer to the release notes for each version
| Package | Python | Platform | Notes |
|---------|--------|----------|-------|
| `ai-dynamo-runtime==1.0.0` | `3.10`–`3.12` | Linux (glibc `v2.28+`) | |
| `ai-dynamo-runtime==0.9.1` | `3.10`–`3.12` | Linux (glibc `v2.28+`) | |
| `ai-dynamo-runtime==0.9.0` | `3.10`–`3.12` | Linux (glibc `v2.28+`) | |
| `ai-dynamo-runtime==0.8.1.post3` | `3.10`–`3.12` | Linux (glibc `v2.28+`) | TRT-LLM `v1.2.0rc6.post3` |
......@@ -303,6 +325,7 @@ For a complete list of known issues, refer to the release notes for each version
| Package | Python | Platform | Notes |
|---------|--------|----------|-------|
| `kvbm==1.0.0` | `3.12` | Linux (glibc `v2.28+`) | |
| `kvbm==0.9.1` | `3.12` | Linux (glibc `v2.28+`) | |
| `kvbm==0.9.0` | `3.12` | Linux (glibc `v2.28+`) | |
| `kvbm==0.8.1` | `3.12` | Linux (glibc `v2.28+`) | |
......@@ -316,11 +339,13 @@ For a complete list of known issues, refer to the release notes for each version
>
> Direct download: `https://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/{chart}-{version}.tgz`
#### dynamo-crds (Helm chart)
#### dynamo-crds (Helm chart) -- Deprecated
> **Note:** The `dynamo-crds` Helm chart is deprecated as of v1.0.0. CRDs are now managed by the Dynamo Operator.
| Chart | Notes |
|-------|-------|
| `dynamo-crds-0.9.1` | |
| `dynamo-crds-0.9.1` | Last release |
| `dynamo-crds-0.9.0` | |
| `dynamo-crds-0.8.1` | |
| `dynamo-crds-0.8.0` | |
......@@ -333,6 +358,7 @@ For a complete list of known issues, refer to the release notes for each version
| Chart | Notes |
|-------|-------|
| `dynamo-platform-1.0.0` | |
| `dynamo-platform-0.9.1` | |
| `dynamo-platform-0.9.0-post1` | Helm fix: operator image tag |
| `dynamo-platform-0.9.0` | |
......@@ -343,6 +369,12 @@ For a complete list of known issues, refer to the release notes for each version
| `dynamo-platform-0.6.1` | |
| `dynamo-platform-0.6.0` | |
#### snapshot (Helm chart)
| Chart | Notes |
|-------|-------|
| `snapshot-1.0.0` | Preview |
#### dynamo-graph (Helm chart) -- Deprecated
> **Note:** The `dynamo-graph` Helm chart is deprecated as of v0.9.0.
......@@ -366,6 +398,7 @@ For a complete list of known issues, refer to the release notes for each version
| Crate | MSRV (Rust) | Notes |
|-------|-------------|-------|
| `dynamo-runtime@1.0.0` | `v1.82` | |
| `dynamo-runtime@0.9.1` | `v1.82` | |
| `dynamo-runtime@0.9.0` | `v1.82` | |
| `dynamo-runtime@0.8.1` | `v1.82` | |
......@@ -379,6 +412,7 @@ For a complete list of known issues, refer to the release notes for each version
| Crate | MSRV (Rust) | Notes |
|-------|-------------|-------|
| `dynamo-llm@1.0.0` | `v1.82` | |
| `dynamo-llm@0.9.1` | `v1.82` | |
| `dynamo-llm@0.9.0` | `v1.82` | |
| `dynamo-llm@0.8.1` | `v1.82` | |
......@@ -392,6 +426,7 @@ For a complete list of known issues, refer to the release notes for each version
| Crate | MSRV (Rust) | Notes |
|-------|-------------|-------|
| `dynamo-async-openai@1.0.0` | `v1.82` | |
| `dynamo-async-openai@0.9.1` | `v1.82` | |
| `dynamo-async-openai@0.9.0` | `v1.82` | |
| `dynamo-async-openai@0.8.1` | `v1.82` | |
......@@ -405,6 +440,7 @@ For a complete list of known issues, refer to the release notes for each version
| Crate | MSRV (Rust) | Notes |
|-------|-------------|-------|
| `dynamo-parsers@1.0.0` | `v1.82` | |
| `dynamo-parsers@0.9.1` | `v1.82` | |
| `dynamo-parsers@0.9.0` | `v1.82` | |
| `dynamo-parsers@0.8.1` | `v1.82` | |
......@@ -418,6 +454,7 @@ For a complete list of known issues, refer to the release notes for each version
| Crate | MSRV (Rust) | Notes |
|-------|-------------|-------|
| `dynamo-memory@1.0.0` | `v1.82` | |
| `dynamo-memory@0.9.1` | `v1.82` | |
| `dynamo-memory@0.9.0` | `v1.82` | |
| `dynamo-memory@0.8.1` | `v1.82` | |
......@@ -427,6 +464,7 @@ For a complete list of known issues, refer to the release notes for each version
| Crate | MSRV (Rust) | Notes |
|-------|-------------|-------|
| `dynamo-config@1.0.0` | `v1.82` | |
| `dynamo-config@0.9.1` | `v1.82` | |
| `dynamo-config@0.9.0` | `v1.82` | |
| `dynamo-config@0.8.1` | `v1.82` | |
......@@ -436,5 +474,18 @@ For a complete list of known issues, refer to the release notes for each version
| Crate | MSRV (Rust) | Notes |
|-------|-------------|-------|
| `dynamo-tokens@1.0.0` | `v1.82` | |
| `dynamo-tokens@0.9.1` | `v1.82` | |
| `dynamo-tokens@0.9.0` | `v1.82` | Initial |
#### dynamo-mocker (crate)
| Crate | MSRV (Rust) | Notes |
|-------|-------------|-------|
| `dynamo-mocker@1.0.0` | `v1.82` | Initial |
#### dynamo-kv-router (crate)
| Crate | MSRV (Rust) | Notes |
|-------|-------------|-------|
| `dynamo-kv-router@1.0.0` | `v1.82` | Initial |
docs/reference/support-matrix.md
View file @ 24fe435d
......@@ -9,7 +9,7 @@ subtitle: Hardware, software, and build compatibility for Dynamo
## At a Glance
**Latest stable release:** [v0.9.1](https://github.com/ai-dynamo/dynamo/releases/tag/v0.9.1) -- SGLang `0.5.8` | TensorRT-LLM `1.3.0rc3` | vLLM `0.14.1` | NIXL `0.9.0`
**Latest stable release:** [v1.0.0](https://github.com/ai-dynamo/dynamo/releases/tag/v1.0.0) -- SGLang `0.5.9` | TensorRT-LLM `1.3.0rc5.post1` | vLLM `0.16.0` | NIXL `0.10.1`
| Requirement | Supported |
| :--- | :--- |
......@@ -28,7 +28,7 @@ The following table shows the backend framework versions included with each Dyna
| **Dynamo** | **SGLang** | **TensorRT-LLM** | **vLLM** | **NIXL** |
| :--- | :--- | :--- | :--- | :--- |
| **main (ToT)** | `0.5.9` | `1.3.0rc5.post1` | `0.16.0` | `0.10.1` |
| **v1.0.0** *(in progress)* | `0.5.9` | `1.3.0rc5.post1` | `0.16.0` | `0.10.1` |
| **v1.0.0** | `0.5.9` | `1.3.0rc5.post1` | `0.16.0` | `0.10.1` |
| **v0.9.1** | `0.5.8` | `1.3.0rc3` | `0.14.1` | `0.9.0` |
| **v0.9.0** | `0.5.8` | `1.3.0rc1` | `0.14.1` | `0.9.0` |
| **v0.8.1.post3** | `0.5.6.post2` | `1.2.0rc6.post3` | `0.12.0` | `0.8.0` |
......@@ -59,7 +59,7 @@ Dynamo container images include CUDA toolkit libraries. The host machine must ha
| Dynamo Version | Backend | CUDA Toolkit | Min Driver | Notes |
| :--- | :--- | :--- | :--- | :--- |
| **1.0.0** *(in progress)* | **SGLang** | 12.9 | 575.xx+ | |
| **1.0.0** | **SGLang** | 12.9 | 575.xx+ | |
| | | 13.0 | 580.xx+ | |
| | **TensorRT-LLM** | 13.1 | 580.xx+ | |
| | **vLLM** | 12.9 | 575.xx+ | |
......@@ -168,13 +168,18 @@ For version-specific artifact details, installation commands, and release histor
- [SGLang Runtime](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/sglang-runtime)
- [SGLang Runtime (CUDA 13)](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/sglang-runtime-cu13)
- [TensorRT-LLM Runtime](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/tensorrtllm-runtime)
- [TensorRT-LLM Runtime (EFA)](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/tensorrtllm-runtime) *(New in v1.0.0, Experimental, AMD64 only)*
- [vLLM Runtime](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/vllm-runtime)
- [vLLM Runtime (CUDA 13)](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/vllm-runtime-cu13)
- [vLLM Runtime (EFA)](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/vllm-runtime) *(New in v1.0.0, Experimental, AMD64 only)*
- [Kubernetes Operator](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/kubernetes-operator)
- [Snapshot Agent](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/snapshot-agent) *(New in v1.0.0, Preview)*
- **Helm Charts**: [NGC](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/collections/ai-dynamo) hosts the helm charts supporting Kubernetes deployments of Dynamo:
- [Dynamo Platform](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/helm-charts/dynamo-platform) (includes CRDs)
- [Dynamo Graph](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/helm-charts/dynamo-graph)
- [Dynamo Platform](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/helm-charts/dynamo-platform) (now includes CRDs)
- [Snapshot](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/helm-charts/snapshot) *(New in v1.0.0, Preview)*
- [Dynamo CRDs](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/helm-charts/dynamo-crds) *(Deprecated in v1.0.0, CRDs managed by Operator)*
- [Dynamo Graph](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/helm-charts/dynamo-graph) *(Deprecated in v0.9.0)*
- **Rust Crates**:
- [dynamo-runtime](https://crates.io/crates/dynamo-runtime/)
......@@ -183,5 +188,8 @@ For version-specific artifact details, installation commands, and release histor
- [dynamo-parsers](https://crates.io/crates/dynamo-parsers/)
- [dynamo-config](https://crates.io/crates/dynamo-config/) *(New in v0.8.0)*
- [dynamo-memory](https://crates.io/crates/dynamo-memory/) *(New in v0.8.0)*
- [dynamo-tokens](https://crates.io/crates/dynamo-tokens/) *(New in v0.9.0)*
- [dynamo-mocker](https://crates.io/crates/dynamo-mocker/) *(New in v1.0.0)*
- [dynamo-kv-router](https://crates.io/crates/dynamo-kv-router/) *(New in v1.0.0)*
Once you've confirmed that your platform and architecture are compatible, you can install **Dynamo** by following the [Local Quick Start](https://github.com/ai-dynamo/dynamo/blob/main/README.md#local-quick-start) in the README.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment