Skip to content

GitLab

Unverified Commit 34f13a12 authored by dagil-nvidia's avatar dagil-nvidia Committed by GitHub
Browse files

docs: use GitHub admonitions instead of Fern-native callouts (#7370) 


Signed-off-by: default avatarDan Gil <dagil@nvidia.com>
parent da783ce0
Hide whitespace changes
Inline Side-by-side
Showing with 46 additions and 47 deletions
docs/getting-started/quickstart.md
View file @ 34f13a12
......@@ -6,11 +6,10 @@ title: Quickstart
This guide covers running Dynamo **using the CLI on your local machine or VM**.
<Info>
**Looking to deploy on Kubernetes instead?**
See the [Kubernetes Installation Guide](../kubernetes/installation-guide.md)
and [Kubernetes Quickstart](../kubernetes/README.md) for cluster deployments.
</Info>
> [!IMPORTANT]
> **Looking to deploy on Kubernetes instead?**
> See the [Kubernetes Installation Guide](../kubernetes/installation-guide.md)
> and [Kubernetes Quickstart](../kubernetes/README.md) for cluster deployments.
## Install Dynamo
......@@ -29,12 +28,11 @@ docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/tensorrtl
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.0.0
```
<Tip>
To run frontend and worker in the same container, either:
- Run processes in background with `&` (see Run Dynamo section below), or
- Open a second terminal and use `docker exec -it <container_id> bash`
</Tip>
> [!TIP]
> To run frontend and worker in the same container, either:
>
> - Run processes in background with `&` (see Run Dynamo section below), or
> - Open a second terminal and use `docker exec -it <container_id> bash`
See [Release Artifacts](../reference/release-artifacts.md#container-images) for available
versions and backend guides for run instructions: [SGLang](../backends/sglang/README.md) |
......@@ -61,10 +59,9 @@ sudo apt install python3-dev
uv pip install --prerelease=allow "ai-dynamo[sglang]"
```
<Note>
For CUDA 13 (B300/GB300), the container is recommended. See
[SGLang install docs](https://docs.sglang.io/get_started/install.html) for details.
</Note>
> [!NOTE]
> For CUDA 13 (B300/GB300), the container is recommended. See
> [SGLang install docs](https://docs.sglang.io/get_started/install.html) for details.
**TensorRT-LLM**
......@@ -74,12 +71,11 @@ pip install torch==2.9.0 torchvision --index-url https://download.pytorch.org/wh
pip install --pre --extra-index-url https://pypi.nvidia.com "ai-dynamo[trtllm]"
```
<Note>
TensorRT-LLM requires `pip` due to a transitive Git URL dependency that
`uv` doesn't resolve. We recommend using the TensorRT-LLM container for
broader compatibility. See the [TRT-LLM backend guide](../backends/trtllm/README.md)
for details.
</Note>
> [!NOTE]
> TensorRT-LLM requires `pip` due to a transitive Git URL dependency that
> `uv` doesn't resolve. We recommend using the TensorRT-LLM container for
> broader compatibility. See the [TRT-LLM backend guide](../backends/trtllm/README.md)
> for details.
**vLLM**
......@@ -90,17 +86,15 @@ uv pip install --prerelease=allow "ai-dynamo[vllm]"
## Run Dynamo
<Tip>
**(Optional)** Before running Dynamo, verify your system configuration:
`python3 deploy/sanity_check.py`
</Tip>
> [!TIP]
> **(Optional)** Before running Dynamo, verify your system configuration:
> `python3 deploy/sanity_check.py`
Start the frontend, then start a worker for your chosen backend.
<Tip>
To run in a single terminal (useful in containers), append `> logfile.log 2>&1 &`
to run processes in background. Example: `python3 -m dynamo.frontend --discovery-backend file > dynamo.frontend.log 2>&1 &`
</Tip>
> [!TIP]
> To run in a single terminal (useful in containers), append `> logfile.log 2>&1 &`
> to run processes in background. Example: `python3 -m dynamo.frontend --discovery-backend file > dynamo.frontend.log 2>&1 &`
```bash
# Start the OpenAI compatible frontend (default port is 8000)
......@@ -129,20 +123,18 @@ python3 -m dynamo.vllm --model Qwen/Qwen3-0.6B --discovery-backend file \
--kv-events-config '{"enable_kv_cache_events": false}'
```
<Note>
For dependency-free local development, disable KV event publishing (avoids NATS):
- **vLLM:** Add `--kv-events-config '{"enable_kv_cache_events": false}'`
- **SGLang:** No flag needed (KV events disabled by default)
- **TensorRT-LLM:** No flag needed (KV events disabled by default)
**TensorRT-LLM only:** The warning `Cannot connect to ModelExpress server/transport error. Using direct download.`
is expected and can be safely ignored.
</Note>
<Note>
**Deprecation notice:** vLLM automatically enables KV event publishing when prefix caching is active. In a future release, this will change — KV events will be disabled by default for all backends. Start using `--kv-events-config` explicitly to prepare.
</Note>
> [!NOTE]
> For dependency-free local development, disable KV event publishing (avoids NATS):
>
> - **vLLM:** Add `--kv-events-config '{"enable_kv_cache_events": false}'`
> - **SGLang:** No flag needed (KV events disabled by default)
> - **TensorRT-LLM:** No flag needed (KV events disabled by default)
>
> **TensorRT-LLM only:** The warning `Cannot connect to ModelExpress server/transport error. Using direct download.`
> is expected and can be safely ignored.
> [!NOTE]
> **Deprecation notice:** vLLM automatically enables KV event publishing when prefix caching is active. In a future release, this will change — KV events will be disabled by default for all backends. Start using `--kv-events-config` explicitly to prepare.
## Test Your Deployment
......
docs/reference/release-artifacts.md
View file @ 34f13a12
......@@ -50,7 +50,8 @@ We recommend using the TensorRT-LLM NGC container instead of the `ai-dynamo[trtl
| `dynamo-platform-1.0.0` | Platform services (etcd, NATS) and Dynamo Operator for Dynamo cluster | [link](https://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-platform-1.0.0.tgz) |
| `snapshot-1.0.0` | Snapshot DaemonSet for fast GPU worker recovery | [link](https://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/snapshot-1.0.0.tgz) |
> **Note:** The `dynamo-crds` Helm chart is deprecated as of v1.0.0; CRDs are now managed by the Dynamo Operator. The `dynamo-graph` Helm chart is deprecated as of v0.9.0.
> [!NOTE]
> The `dynamo-crds` Helm chart is deprecated as of v1.0.0; CRDs are now managed by the Dynamo Operator. The `dynamo-graph` Helm chart is deprecated as of v0.9.0.
### Rust Crates
......@@ -70,7 +71,8 @@ We recommend using the TensorRT-LLM NGC container instead of the `ai-dynamo[trtl
### Container Images (NGC)
> For detailed run instructions, see the [Container README](https://github.com/ai-dynamo/dynamo/tree/main/container/README.md) or backend-specific guides: [vLLM](../backends/vllm/README.md) | [SGLang](../backends/sglang/README.md) | [TensorRT-LLM](../backends/trtllm/README.md)
> [!TIP]
> For detailed run instructions, see the backend-specific guides: [vLLM](../backends/vllm/README.md) | [SGLang](../backends/sglang/README.md) | [TensorRT-LLM](../backends/trtllm/README.md)
```bash
# Runtime containers
......@@ -94,6 +96,7 @@ docker pull nvcr.io/nvidia/ai-dynamo/snapshot-agent:1.0.0
### Python Wheels (PyPI)
> [!TIP]
> For detailed installation instructions, see the [Local Quick Start](https://github.com/ai-dynamo/dynamo#local-quick-start) in the README.
```bash
......@@ -112,6 +115,7 @@ uv pip install kvbm==1.0.0
### Helm Charts (NGC)
> [!TIP]
> For Kubernetes deployment instructions, see the [Kubernetes Installation Guide](../kubernetes/installation-guide.md).
```bash
......@@ -121,6 +125,7 @@ helm install snapshot oci://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/snapshot
### Rust Crates (crates.io)
> [!TIP]
> For API documentation, see each crate on [docs.rs](https://docs.rs/). To build Dynamo from source, see [Building from Source](https://github.com/ai-dynamo/dynamo#building-from-source).
```bash
......@@ -341,7 +346,8 @@ For a complete list of known issues, refer to the release notes for each version
#### dynamo-crds (Helm chart) -- Deprecated
> **Note:** The `dynamo-crds` Helm chart is deprecated as of v1.0.0. CRDs are now managed by the Dynamo Operator.
> [!NOTE]
> The `dynamo-crds` Helm chart is deprecated as of v1.0.0. CRDs are now managed by the Dynamo Operator.
| Chart | Notes |
|-------|-------|
......@@ -377,7 +383,8 @@ For a complete list of known issues, refer to the release notes for each version
#### dynamo-graph (Helm chart) -- Deprecated
> **Note:** The `dynamo-graph` Helm chart is deprecated as of v0.9.0.
> [!NOTE]
> The `dynamo-graph` Helm chart is deprecated as of v0.9.0.
| Chart | Notes |
|-------|-------|
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment