Unverified Commit ac020629 authored by Dillon Cullinan's avatar Dillon Cullinan Committed by GitHub
Browse files

feat: Dockerfile templating (#5633)


Signed-off-by: default avatarDillon Cullinan <dcullinan@nvidia.com>
parent 5755a8de
...@@ -74,7 +74,8 @@ docker compose -f deploy/docker-compose.yml up -d ...@@ -74,7 +74,8 @@ docker compose -f deploy/docker-compose.yml up -d
We have public images available on [NGC Catalog](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/collections/ai-dynamo/artifacts). If you'd like to build your own container from source: We have public images available on [NGC Catalog](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/collections/ai-dynamo/artifacts). If you'd like to build your own container from source:
```bash ```bash
./container/build.sh --framework VLLM python container/render.py --framework vllm --short-output
docker build -f container/rendered.Dockerfile -t dynamo:latest-vllm .
``` ```
### Run container ### Run container
......
...@@ -42,8 +42,9 @@ See the [support matrix](../../../docs/reference/support-matrix.md) for version ...@@ -42,8 +42,9 @@ See the [support matrix](../../../docs/reference/support-matrix.md) for version
The pip wheel is built through a Docker build process: The pip wheel is built through a Docker build process:
```bash ```bash
# Build the Docker image with KVBM enabled (from the dynamo repo root) # Render and build the Docker image with KVBM enabled (from the dynamo repo root)
./container/build.sh --framework none --enable-kvbm --tag local-kvbm python container/render.py --framework=dynamo --target==runtime --short-output
docker build --build-arg ENABLE_KVBM="true" -f container/rendered.Dockerfile .
``` ```
Once built, you can either: Once built, you can either:
......
...@@ -50,7 +50,7 @@ register_llm( ...@@ -50,7 +50,7 @@ register_llm(
> **Requires GPU node**: The frontend must run on a node with GPU access. During media processing, decoded tensors are written to GPU memory via NIXL, which requires `libcuda.so.1` to be available. Running the frontend on a CPU-only node will fail with something like: `Failed to initialize required backends: [UCX: No UCX plugin found]`. > **Requires GPU node**: The frontend must run on a node with GPU access. During media processing, decoded tensors are written to GPU memory via NIXL, which requires `libcuda.so.1` to be available. Running the frontend on a CPU-only node will fail with something like: `Failed to initialize required backends: [UCX: No UCX plugin found]`.
> [!WARNING] > [!WARNING]
> **Video decoding**: Video decoding needs to be enabled via the `dynamo-llm/media-ffmpeg` rust feature. The following ffmpeg dynamic libraries must be available on the system: `libavcodec`, `libavdevice`, `libavfilter`, `libavformat`, `libswresample`, `libswscale`. These are available in dynamo containers built with `container/build.sh --enable-media-ffmpeg ...` > **Video decoding**: Video decoding needs to be enabled via the `dynamo-llm/media-ffmpeg` rust feature. The following ffmpeg dynamic libraries must be available on the system: `libavcodec`, `libavdevice`, `libavfilter`, `libavformat`, `libswresample`, `libswscale`. These are available in dynamo dockerfiles rendered with `enable_media_ffmpeg` set to true in `container/context.yaml`.
## Image decoding options ## Image decoding options
......
...@@ -129,7 +129,8 @@ then this test will be included in the nightly run as well. ...@@ -129,7 +129,8 @@ then this test will be included in the nightly run as well.
``` ```
- Run in container: - Run in container:
```bash ```bash
./container/build.sh --framework <backend> python container/render.py --framework=${FRAMEWORK} --target=runtime --short-output
docker build -t dynamo:${FRAMEWORK}-latest -f container/rendered.Dockerfile .
./container/run.sh --mount-workspace -it -- pytest ./container/run.sh --mount-workspace -it -- pytest
./container/run.sh --mount-workspace -it -- pytest -m [optional markers] ./container/run.sh --mount-workspace -it -- pytest -m [optional markers]
``` ```
......
This diff is collapsed.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment