Also available: [`tensorrtllm-runtime:1.0.0`](https://docs.nvidia.com/dynamo/resources/release-artifacts) and [`vllm-runtime:1.0.0`](https://docs.nvidia.com/dynamo/resources/release-artifacts).
Also available: [`tensorrtllm-runtime:1.0.1`](https://docs.nvidia.com/dynamo/resources/release-artifacts) and [`vllm-runtime:1.0.1`](https://docs.nvidia.com/dynamo/resources/release-artifacts).
> **Experimental:** [v1.1.0-dev.1](#pre-release-artifacts) is available as an experimental preview. See [Pre-Release Artifacts](#pre-release-artifacts) for container images, wheels, and Helm charts.
### Container Images
| Image:Tag | Description | Backend | CUDA | Arch | NGC | Notes |
| `dynamo-frontend:1.0.1` | API gateway with Endpoint Prediction Protocol (EPP) | — | — | AMD64/ARM64 | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/dynamo-frontend?version=1.0.1) | |
| `kubernetes-operator:1.0.1` | Kubernetes operator for Dynamo deployments | — | — | AMD64/ARM64 | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/kubernetes-operator?version=1.0.1) | |
| `snapshot-agent:1.0.1` | Snapshot agent for fast GPU worker recovery via CRIU | — | — | AMD64/ARM64 | [link](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/containers/snapshot-agent?version=1.0.1) | Preview |
\* Multimodal inference on CUDA 13 images: works on AMD64 for all backends; works on ARM64 only for TensorRT-LLM (`vllm-runtime:*-cuda13` and `sglang-runtime:*-cuda13` do not support multimodality on ARM64).
...
...
@@ -39,16 +41,16 @@ We recommend using the TensorRT-LLM NGC container instead of the `ai-dynamo[trtl
| `ai-dynamo==1.0.0` | Main package with backend integrations (vLLM, SGLang, TRT-LLM) | `3.10`–`3.12` | Linux (glibc `v2.28+`) | [link](https://pypi.org/project/ai-dynamo/1.0.0/) |
| `ai-dynamo-runtime==1.0.0` | Core Python bindings for Dynamo runtime | `3.10`–`3.12` | Linux (glibc `v2.28+`) | [link](https://pypi.org/project/ai-dynamo-runtime/1.0.0/) |
| `kvbm==1.0.0` | KV Block Manager for disaggregated KV cache | `3.12` | Linux (glibc `v2.28+`) | [link](https://pypi.org/project/kvbm/1.0.0/) |
| `ai-dynamo==1.0.1` | Main package with backend integrations (vLLM, SGLang, TRT-LLM) | `3.10`–`3.12` | Linux (glibc `v2.28+`) | [link](https://pypi.org/project/ai-dynamo/1.0.1/) |
| `ai-dynamo-runtime==1.0.1` | Core Python bindings for Dynamo runtime | `3.10`–`3.12` | Linux (glibc `v2.28+`) | [link](https://pypi.org/project/ai-dynamo-runtime/1.0.1/) |
| `kvbm==1.0.1` | KV Block Manager for disaggregated KV cache | `3.12` | Linux (glibc `v2.28+`) | [link](https://pypi.org/project/kvbm/1.0.1/) |
### Helm Charts
| Chart | Description | NGC |
|-------|-------------|-----|
| `dynamo-platform-1.0.0` | Platform services (etcd, NATS) and Dynamo Operator for Dynamo cluster | [link](https://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-platform-1.0.0.tgz) |
| `snapshot-1.0.0` | Snapshot DaemonSet for fast GPU worker recovery | [link](https://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/snapshot-1.0.0.tgz) |
| `dynamo-platform-1.0.1` | Platform services (etcd, NATS) and Dynamo Operator for Dynamo cluster | [link](https://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-platform-1.0.1.tgz) |
| `snapshot-1.0.1` | Snapshot DaemonSet for fast GPU worker recovery | [link](https://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/snapshot-1.0.1.tgz) |
> [!NOTE]
> The `dynamo-crds` Helm chart is deprecated as of v1.0.0; CRDs are now managed by the Dynamo Operator. The `dynamo-graph` Helm chart is deprecated as of v0.9.0.
...
...
@@ -57,15 +59,15 @@ We recommend using the TensorRT-LLM NGC container instead of the `ai-dynamo[trtl
> For API documentation, see each crate on [docs.rs](https://docs.rs/). To build Dynamo from source, see [Building from Source](https://github.com/ai-dynamo/dynamo#building-from-source).
```bash
cargo add dynamo-runtime@1.0.0
cargo add dynamo-llm@1.0.0
cargo add dynamo-async-openai@1.0.0
cargo add dynamo-parsers@1.0.0
cargo add dynamo-memory@1.0.0
cargo add dynamo-config@1.0.0
cargo add dynamo-tokens@1.0.0
cargo add dynamo-mocker@1.0.0
cargo add dynamo-kv-router@1.0.0
cargo add dynamo-runtime@1.0.1
cargo add dynamo-llm@1.0.1
cargo add dynamo-async-openai@1.0.1
cargo add dynamo-parsers@1.0.1
cargo add dynamo-memory@1.0.1
cargo add dynamo-config@1.0.1
cargo add dynamo-tokens@1.0.1
cargo add dynamo-mocker@1.0.1
cargo add dynamo-kv-router@1.0.1
```
**CUDA and Driver Requirements:** For detailed CUDA toolkit versions and minimum driver requirements for each container image, see the [Support Matrix](support-matrix.md#cuda-and-driver-requirements).
@@ -162,6 +165,8 @@ For a complete list of known issues, refer to the release notes for each version
## Release History
-**v1.1.0-dev.1***(experimental)*: Preview release. SGLang `v0.5.9`, TRT-LLM `v1.3.0rc5.post1`, vLLM `v0.17.1`, NIXL `v0.10.1`. Not recommended for production use.
-**v1.0.1**: Patch release. Same backend versions as v1.0.0: SGLang `v0.5.9`, TRT-LLM `v1.3.0rc5.post1`, vLLM `v0.16.0`, NIXL `v0.10.1`.
-**v1.0.0**: First major release. SGLang `v0.5.9`, TRT-LLM `v1.3.0rc5.post1` (CUDA 13.1), vLLM `v0.16.0`, NIXL `v0.10.1`. New `snapshot-agent` container and `snapshot` Helm chart (Preview). New EFA container variants for vLLM and TRT-LLM (Experimental, AMD64 only). New `dynamo-mocker` and `dynamo-kv-router` Rust crates. Deprecated `dynamo-crds` Helm chart (CRDs now managed by the Operator). `v1alpha1` CRDs deprecated.
-**v0.9.1**: Updated TRT-LLM to `v1.3.0rc3`. All other backend versions unchanged from v0.9.0.
@@ -298,6 +315,7 @@ For a complete list of known issues, refer to the release notes for each version
| Package | Python | Platform | Notes |
|---------|--------|----------|-------|
| `ai-dynamo==1.0.1` | `3.10`–`3.12` | Linux (glibc `v2.28+`) | |
| `ai-dynamo==1.0.0` | `3.10`–`3.12` | Linux (glibc `v2.28+`) | |
| `ai-dynamo==0.9.1` | `3.10`–`3.12` | Linux (glibc `v2.28+`) | |
| `ai-dynamo==0.9.0` | `3.10`–`3.12` | Linux (glibc `v2.28+`) | |
...
...
@@ -314,6 +332,7 @@ For a complete list of known issues, refer to the release notes for each version
| Package | Python | Platform | Notes |
|---------|--------|----------|-------|
| `ai-dynamo-runtime==1.0.1` | `3.10`–`3.12` | Linux (glibc `v2.28+`) | |
| `ai-dynamo-runtime==1.0.0` | `3.10`–`3.12` | Linux (glibc `v2.28+`) | |
| `ai-dynamo-runtime==0.9.1` | `3.10`–`3.12` | Linux (glibc `v2.28+`) | |
| `ai-dynamo-runtime==0.9.0` | `3.10`–`3.12` | Linux (glibc `v2.28+`) | |
...
...
@@ -330,6 +349,7 @@ For a complete list of known issues, refer to the release notes for each version
| Package | Python | Platform | Notes |
|---------|--------|----------|-------|
| `kvbm==1.0.1` | `3.12` | Linux (glibc `v2.28+`) | |
| `kvbm==1.0.0` | `3.12` | Linux (glibc `v2.28+`) | |
| `kvbm==0.9.1` | `3.12` | Linux (glibc `v2.28+`) | |
| `kvbm==0.9.0` | `3.12` | Linux (glibc `v2.28+`) | |
...
...
@@ -364,6 +384,7 @@ For a complete list of known issues, refer to the release notes for each version
| Chart | Notes |
|-------|-------|
| `dynamo-platform-1.0.1` | |
| `dynamo-platform-1.0.0` | |
| `dynamo-platform-0.9.1` | |
| `dynamo-platform-0.9.0-post1` | Helm fix: operator image tag |
...
...
@@ -379,6 +400,7 @@ For a complete list of known issues, refer to the release notes for each version
| Chart | Notes |
|-------|-------|
| `snapshot-1.0.1` | Preview |
| `snapshot-1.0.0` | Preview |
#### dynamo-graph (Helm chart) -- Deprecated
...
...
@@ -405,6 +427,7 @@ For a complete list of known issues, refer to the release notes for each version
| Crate | MSRV (Rust) | Notes |
|-------|-------------|-------|
| `dynamo-runtime@1.0.1` | `v1.82` | |
| `dynamo-runtime@1.0.0` | `v1.82` | |
| `dynamo-runtime@0.9.1` | `v1.82` | |
| `dynamo-runtime@0.9.0` | `v1.82` | |
...
...
@@ -419,6 +442,7 @@ For a complete list of known issues, refer to the release notes for each version
| Crate | MSRV (Rust) | Notes |
|-------|-------------|-------|
| `dynamo-llm@1.0.1` | `v1.82` | |
| `dynamo-llm@1.0.0` | `v1.82` | |
| `dynamo-llm@0.9.1` | `v1.82` | |
| `dynamo-llm@0.9.0` | `v1.82` | |
...
...
@@ -433,6 +457,7 @@ For a complete list of known issues, refer to the release notes for each version
| Crate | MSRV (Rust) | Notes |
|-------|-------------|-------|
| `dynamo-async-openai@1.0.1` | `v1.82` | |
| `dynamo-async-openai@1.0.0` | `v1.82` | |
| `dynamo-async-openai@0.9.1` | `v1.82` | |
| `dynamo-async-openai@0.9.0` | `v1.82` | |
...
...
@@ -447,6 +472,7 @@ For a complete list of known issues, refer to the release notes for each version
| Crate | MSRV (Rust) | Notes |
|-------|-------------|-------|
| `dynamo-parsers@1.0.1` | `v1.82` | |
| `dynamo-parsers@1.0.0` | `v1.82` | |
| `dynamo-parsers@0.9.1` | `v1.82` | |
| `dynamo-parsers@0.9.0` | `v1.82` | |
...
...
@@ -461,6 +487,7 @@ For a complete list of known issues, refer to the release notes for each version
| Crate | MSRV (Rust) | Notes |
|-------|-------------|-------|
| `dynamo-memory@1.0.1` | `v1.82` | |
| `dynamo-memory@1.0.0` | `v1.82` | |
| `dynamo-memory@0.9.1` | `v1.82` | |
| `dynamo-memory@0.9.0` | `v1.82` | |
...
...
@@ -471,6 +498,7 @@ For a complete list of known issues, refer to the release notes for each version
| Crate | MSRV (Rust) | Notes |
|-------|-------------|-------|
| `dynamo-config@1.0.1` | `v1.82` | |
| `dynamo-config@1.0.0` | `v1.82` | |
| `dynamo-config@0.9.1` | `v1.82` | |
| `dynamo-config@0.9.0` | `v1.82` | |
...
...
@@ -481,6 +509,7 @@ For a complete list of known issues, refer to the release notes for each version
| Crate | MSRV (Rust) | Notes |
|-------|-------------|-------|
| `dynamo-tokens@1.0.1` | `v1.82` | |
| `dynamo-tokens@1.0.0` | `v1.82` | |
| `dynamo-tokens@0.9.1` | `v1.82` | |
| `dynamo-tokens@0.9.0` | `v1.82` | Initial |
...
...
@@ -489,10 +518,61 @@ For a complete list of known issues, refer to the release notes for each version
| Crate | MSRV (Rust) | Notes |
|-------|-------------|-------|
| `dynamo-mocker@1.0.1` | `v1.82` | |
| `dynamo-mocker@1.0.0` | `v1.82` | Initial |
#### dynamo-kv-router (crate)
| Crate | MSRV (Rust) | Notes |
|-------|-------------|-------|
| `dynamo-kv-router@1.0.1` | `v1.82` | |
| `dynamo-kv-router@1.0.0` | `v1.82` | Initial |
---
## Pre-Release Artifacts
> [!WARNING]
> **Pre-Release artifacts do not go through QA validation.** Pre-release versions are experimental previews intended for early testing and feedback. They may contain bugs, breaking changes, or incomplete features. Use stable releases for production workloads.
