Unverified Commit 3188c70a authored by Dillon Cullinan's avatar Dillon Cullinan Committed by GitHub
Browse files

chore: Templating Feedback Followup (#6125)


Signed-off-by: default avatarDillon Cullinan <dcullinan@nvidia.com>
parent 33ddd8fb
...@@ -2,6 +2,7 @@ ...@@ -2,6 +2,7 @@
# SPDX-FileCopyrightText: Copyright (c) 2024-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-FileCopyrightText: Copyright (c) 2024-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0 # SPDX-License-Identifier: Apache-2.0
#} #}
# === BEGIN templates/trtllm_runtime.Dockerfile ===
################################################## ##################################################
########## Runtime Image ######################## ########## Runtime Image ########################
################################################## ##################################################
......
...@@ -2,6 +2,7 @@ ...@@ -2,6 +2,7 @@
# SPDX-FileCopyrightText: Copyright (c) 2024-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-FileCopyrightText: Copyright (c) 2024-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0 # SPDX-License-Identifier: Apache-2.0
#} #}
# === BEGIN templates/vllm_framework.Dockerfile ===
######################################################## ########################################################
########## Framework Development Image ################ ########## Framework Development Image ################
######################################################## ########################################################
......
...@@ -2,6 +2,7 @@ ...@@ -2,6 +2,7 @@
# SPDX-FileCopyrightText: Copyright (c) 2024-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-FileCopyrightText: Copyright (c) 2024-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0 # SPDX-License-Identifier: Apache-2.0
#} #}
# === BEGIN templates/vllm_runtime.Dockerfile ===
################################################## ##################################################
########## Runtime Image ######################## ########## Runtime Image ########################
################################################## ##################################################
......
...@@ -2,6 +2,7 @@ ...@@ -2,6 +2,7 @@
# SPDX-FileCopyrightText: Copyright (c) 2024-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-FileCopyrightText: Copyright (c) 2024-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0 # SPDX-License-Identifier: Apache-2.0
#} #}
# === BEGIN templates/wheel_builder.Dockerfile ===
################################## ##################################
##### Wheel Build Image ########## ##### Wheel Build Image ##########
################################## ##################################
......
...@@ -134,7 +134,7 @@ We are in the process of shipping pre-built docker containers that contain insta ...@@ -134,7 +134,7 @@ We are in the process of shipping pre-built docker containers that contain insta
```bash ```bash
cd $DYNAMO_ROOT cd $DYNAMO_ROOT
python container/render.py --framework=sglang --target=runtime --short-output python container/render.py --framework=sglang --target=runtime --output-short-filename
docker build -t dynamo:sglang-latest -f container/rendered.Dockerfile . docker build -t dynamo:sglang-latest -f container/rendered.Dockerfile .
``` ```
......
...@@ -93,11 +93,11 @@ docker compose -f deploy/docker-compose.yml up -d ...@@ -93,11 +93,11 @@ docker compose -f deploy/docker-compose.yml up -d
apt-get update && apt-get -y install git git-lfs apt-get update && apt-get -y install git git-lfs
# On an x86 machine: # On an x86 machine:
python container/render.py --framework=trtllm --target=runtime --short-output python container/render.py --framework=trtllm --target=runtime --output-short-filename
docker build -t dynamo:trtllm-latest -f container/rendered.Dockerfile . docker build -t dynamo:trtllm-latest -f container/rendered.Dockerfile .
# On an ARM machine: # On an ARM machine:
python container/render.py --framework=trtllm --target=runtime --platform=arm64 --short-output python container/render.py --framework=trtllm --target=runtime --platform=arm64 --output-short-filename
docker build -t dynamo:trtllm-latest -f container/rendered.Dockerfile . docker build -t dynamo:trtllm-latest -f container/rendered.Dockerfile .
``` ```
......
...@@ -74,7 +74,7 @@ docker compose -f deploy/docker-compose.yml up -d ...@@ -74,7 +74,7 @@ docker compose -f deploy/docker-compose.yml up -d
We have public images available on [NGC Catalog](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/collections/ai-dynamo/artifacts). If you'd like to build your own container from source: We have public images available on [NGC Catalog](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/collections/ai-dynamo/artifacts). If you'd like to build your own container from source:
```bash ```bash
python container/render.py --framework=vllm --target=runtime --short-output python container/render.py --framework=vllm --target=runtime --output-short-filename
docker build -t dynamo:vllm-latest -f container/rendered.Dockerfile . docker build -t dynamo:vllm-latest -f container/rendered.Dockerfile .
``` ```
......
...@@ -39,7 +39,7 @@ From the Dynamo repository root: ...@@ -39,7 +39,7 @@ From the Dynamo repository root:
```bash ```bash
# Build the base Dynamo image # Build the base Dynamo image
python container/render.py --framework=dynamo --target=runtime --short-output python container/render.py --framework=dynamo --target=runtime --output-short-filename
docker build -f container/rendered.Dockerfile . docker build -f container/rendered.Dockerfile .
# Build the Triton worker image # Build the Triton worker image
......
...@@ -112,7 +112,7 @@ Before using these templates, ensure you have: ...@@ -112,7 +112,7 @@ Before using these templates, ensure you have:
The deployment files currently require access to `my-registry/tensorrtllm-runtime`. If you don't have access, build and push your own image: The deployment files currently require access to `my-registry/tensorrtllm-runtime`. If you don't have access, build and push your own image:
```bash ```bash
python container/render.py --framework=trtllm --short-output python container/render.py --framework=trtllm --output-short-filename
docker build -f container/rendered.Dockerfile . docker build -f container/rendered.Dockerfile .
# Tag and push to your container registry # Tag and push to your container registry
# Update the image references in the YAML files # Update the image references in the YAML files
...@@ -125,7 +125,7 @@ apt-get update && apt-get -y install git git-lfs ...@@ -125,7 +125,7 @@ apt-get update && apt-get -y install git git-lfs
For ARM machines, use: For ARM machines, use:
```bash ```bash
python container/render.py --framework=vllm --platform arm64 --short-output python container/render.py --framework=vllm --platform arm64 --output-short-filename
docker build -f container/rendered.Dockerfile . docker build -f container/rendered.Dockerfile .
``` ```
......
...@@ -102,7 +102,7 @@ Before using these templates, ensure you have: ...@@ -102,7 +102,7 @@ Before using these templates, ensure you have:
We have public images available on [NGC Catalog](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/collections/ai-dynamo/artifacts). If you'd prefer to use your own registry, build and push your own image: We have public images available on [NGC Catalog](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/collections/ai-dynamo/artifacts). If you'd prefer to use your own registry, build and push your own image:
```bash ```bash
python container/render.py --framework=vllm --short-output python container/render.py --framework=vllm --output-short-filename
docker build -f container/rendered.Dockerfile . docker build -f container/rendered.Dockerfile .
# Tag and push to your container registry # Tag and push to your container registry
# Update the image references in the YAML files # Update the image references in the YAML files
......
...@@ -17,7 +17,7 @@ export DOCKER_SERVER=<ECR_REGISTRY> ...@@ -17,7 +17,7 @@ export DOCKER_SERVER=<ECR_REGISTRY>
export DOCKER_USERNAME=AWS export DOCKER_USERNAME=AWS
export DOCKER_PASSWORD="$(aws ecr get-login-password --region <ECR_REGION>)" export DOCKER_PASSWORD="$(aws ecr get-login-password --region <ECR_REGION>)"
export IMAGE_TAG=0.3.2.1 export IMAGE_TAG=0.3.2.1
python container/render.py --framework=dynamo --target=runtime --short-output python container/render.py --framework=dynamo --target=runtime --output-short-filename
docker build -t dynamo:latest-vllm -f container/rendered.Dockerfile . docker build -t dynamo:latest-vllm -f container/rendered.Dockerfile .
``` ```
......
...@@ -126,7 +126,7 @@ We are in the process of shipping pre-built docker containers that contain insta ...@@ -126,7 +126,7 @@ We are in the process of shipping pre-built docker containers that contain insta
```bash ```bash
cd $DYNAMO_ROOT cd $DYNAMO_ROOT
python container/render.py --framework sglang --short-output python container/render.py --framework sglang --output-short-filename
docker build -f container/rendered.Dockerfile -t dynamo:latest-sglang . docker build -f container/rendered.Dockerfile -t dynamo:latest-sglang .
``` ```
......
...@@ -80,11 +80,11 @@ docker compose -f deploy/docker-compose.yml up -d ...@@ -80,11 +80,11 @@ docker compose -f deploy/docker-compose.yml up -d
apt-get update && apt-get -y install git git-lfs apt-get update && apt-get -y install git git-lfs
# On an x86 machine: # On an x86 machine:
python container/render.py --framework sglang --short-output python container/render.py --framework sglang --output-short-filename
docker build -f container/rendered.Dockerfile -t dynamo:latest-trtllm . docker build -f container/rendered.Dockerfile -t dynamo:latest-trtllm .
# On an ARM machine: # On an ARM machine:
python container/render.py --framework trtllm --platform arm64 --short-output python container/render.py --framework trtllm --platform arm64 --output-short-filename
docker build -f container/rendered.Dockerfile -t dynamo:latest-trtllm . docker build -f container/rendered.Dockerfile -t dynamo:latest-trtllm .
``` ```
......
...@@ -74,7 +74,7 @@ docker compose -f deploy/docker-compose.yml up -d ...@@ -74,7 +74,7 @@ docker compose -f deploy/docker-compose.yml up -d
We have public images available on [NGC Catalog](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/collections/ai-dynamo/artifacts). If you'd like to build your own container from source: We have public images available on [NGC Catalog](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/collections/ai-dynamo/artifacts). If you'd like to build your own container from source:
```bash ```bash
python container/render.py --framework vllm --short-output python container/render.py --framework vllm --output-short-filename
docker build -f container/rendered.Dockerfile -t dynamo:latest-vllm . docker build -f container/rendered.Dockerfile -t dynamo:latest-vllm .
``` ```
......
...@@ -43,7 +43,7 @@ The pip wheel is built through a Docker build process: ...@@ -43,7 +43,7 @@ The pip wheel is built through a Docker build process:
```bash ```bash
# Render and build the Docker image with KVBM enabled (from the dynamo repo root) # Render and build the Docker image with KVBM enabled (from the dynamo repo root)
python container/render.py --framework=dynamo --target==runtime --short-output python container/render.py --framework=dynamo --target==runtime --output-short-filename
docker build --build-arg ENABLE_KVBM="true" -f container/rendered.Dockerfile . docker build --build-arg ENABLE_KVBM="true" -f container/rendered.Dockerfile .
``` ```
......
...@@ -135,7 +135,7 @@ then this test will be included in the nightly run as well. ...@@ -135,7 +135,7 @@ then this test will be included in the nightly run as well.
``` ```
- Run in container: - Run in container:
```bash ```bash
python container/render.py --framework=${FRAMEWORK} --target=runtime --short-output python container/render.py --framework=${FRAMEWORK} --target=runtime --output-short-filename
docker build -t dynamo:${FRAMEWORK}-latest -f container/rendered.Dockerfile . docker build -t dynamo:${FRAMEWORK}-latest -f container/rendered.Dockerfile .
./container/run.sh --mount-workspace -it -- pytest ./container/run.sh --mount-workspace -it -- pytest
./container/run.sh --mount-workspace -it -- pytest -m [optional markers] ./container/run.sh --mount-workspace -it -- pytest -m [optional markers]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment