Skip to content

GitLab

Unverified Commit 10bfb73a authored by Keiven C's avatar Keiven C Committed by GitHub
Browse files

refactor: simplify Dockerfile.vllm, enable local-dev for all frameworks (#3099) 


Signed-off-by: default avatarKeiven Chang <keivenchang@users.noreply.github.com>
parent 70a8aa3f
Hide whitespace changes
Inline Side-by-side
Showing with 367 additions and 206 deletions
.devcontainer/README.md
View file @ 10bfb73a
...@@ -91,13 +91,21 @@ Follow these steps to get your NVIDIA Dynamo development environment up and runn ...@@ -91,13 +91,21 @@ Follow these steps to get your NVIDIA Dynamo development environment up and runn
### Step 1: Build the Development Container Image ### Step 1: Build the Development Container Image
Build `dynamo:latest-vllm` from scratch from the source: Build `dynamo:latest-vllm-local-dev` from scratch from the source:
```bash ```bash
./container/build.sh --target dev --framework VLLM # Single command approach (recommended)
./container/build.sh --framework VLLM --target local-dev
# Creates both dynamo:latest-vllm and dynamo:latest-vllm-local-dev
# Alternatively, you can build a development container then local-dev
./container/build.sh --framework VLLM
# Now you have a development image dynamo:latest-vllm
./container/build.sh --dev-image dynamo:latest-vllm --framework VLLM
# Now you have a local-dev image dynamo:latest-vllm-local-dev
``` ```
The container will be built and give certain file permissions to your local uid and gid. The local-dev image will give you local user permissions matching your host user and includes extra developer utilities (debugging tools, text editors, system monitors, etc.).
### Step 2: Install Dev Containers Extension ### Step 2: Install Dev Containers Extension
...@@ -235,30 +243,6 @@ cp .devcontainer/devcontainer.json .devcontainer/jensen_dev/devcontainer.json ...@@ -235,30 +243,6 @@ cp .devcontainer/devcontainer.json .devcontainer/jensen_dev/devcontainer.json
Common customizations include additional mounts, environment variables, IDE extensions, and build arguments. When you open a new Dev Container, you can pick from any of the `.devcontainer/<path>/devcontainer.json` files available. Common customizations include additional mounts, environment variables, IDE extensions, and build arguments. When you open a new Dev Container, you can pick from any of the `.devcontainer/<path>/devcontainer.json` files available.
### SGLANG Custom devcontainer.json Configuration (EXPERIMENTAL)
This is experimental. Please update/fix if you encounter problems. For sglang Dev Container, you first need to build `dynamo:latest-sglang-local-dev` image like this (wait about half an hour):
```bash
./container/build.sh --framework SGLANG --target local-dev
```
Then, make a copy of the `devcontainer.json file` to a directory of your choice. For this example, we'll just call it `sglang`:
```bash
mkdir .devcontainer/sglang/
cp -a .devcontainer/devcontainer.json .devcontainer/sglang/
```
Afterwards, edit your `.devcontainer/sglang/devcontainer.json` so that the name and image correspond to SGLANG. Example:
```json
"name": "[sglang] This is my amazing custom Dev Container Development",
...
"image": "dynamo:latest-sglang-local-dev",
```
Now, go to **Dev Containers: Open Folder in Container** and select `[sglang] This is my amazing custom Dev Container Development`. The post-create.sh script should be running.
### SSH Keys for Git Operations ### SSH Keys for Git Operations
...@@ -372,13 +356,15 @@ If you see errors like "container is not running" or "An error occurred setting ...@@ -372,13 +356,15 @@ If you see errors like "container is not running" or "An error occurred setting
**Common Causes and Solutions:** **Common Causes and Solutions:**
1. **Missing base image:** 1. **Missing a local-dev image:**
```bash ```bash
# Check if the required image exists # Check if the required local-dev image exists
docker images | grep dynamo docker images | grep dynamo
# If missing, build the dev image first # If missing, build the dev image first, then build local-dev
./container/build.sh --target local-dev ./container/build.sh --framework vllm
./container/build.sh --dev-image dynamo:latest-vllm --framework vllm
# Output: dynamo:latest-vllm-local-dev
``` ```
2. **Container startup failure:** 2. **Container startup failure:**
......
container/Dockerfile.local_dev 0 → 100644
View file @ 10bfb73a
# syntax=docker/dockerfile:1.10.0
# SPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
# This Dockerfile creates a local development environment for Dev Container plug-in use.
# It takes a BASE image (typically the dev target) and adds local-dev specific configurations
# including additional developer utilities and tools.
#
# Usage:
# - Dev Container IDE Extension: Use directly with VS Code/Cursor Dev Container extension
# - Command line: run.sh --image <local-dev-image> --mount-workspace ...
# where the ubuntu user inside the container is mapped to your local user login
ARG DEV_BASE=""
FROM ${DEV_BASE} AS local-dev
# Don't want ubuntu to be editable, just change uid and gid.
ENV USERNAME=ubuntu
ARG USER_UID
ARG USER_GID
ARG WORKSPACE_DIR=/workspace
ARG ARCH
# Update package lists and install developer utilities. Some of these may exist in the base image,
# but to ensure consistency across all dev images, we explicitly list all required dev tools here.
RUN apt-get update && apt-get install -y \
# Development utilities
curl wget git vim nano \
# System utilities
htop nvtop tmux screen \
# Network utilities
net-tools iproute2 iputils-ping \
# Archive utilities
zip unzip rsync \
# Build tools
build-essential cmake autoconf automake libtool \
# Debug and analysis tools
gdb valgrind strace ltrace \
# Text processing
jq yq grep sed \
# File utilities
tree fd-find ripgrep \
# Shell utilities
zsh fish bash-completion
# https://code.visualstudio.com/remote/advancedcontainers/add-nonroot-user
# Configure user with sudo access for Dev Container workflows
RUN apt-get install -y sudo gnupg2 gnupg1 \
&& echo "$USERNAME ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/$USERNAME \
&& chmod 0440 /etc/sudoers.d/$USERNAME \
&& mkdir -p /home/$USERNAME \
&& groupmod -g $USER_GID $USERNAME \
&& usermod -u $USER_UID -g $USER_GID $USERNAME \
&& chown -R $USERNAME:$USERNAME /home/$USERNAME \
&& chsh -s /bin/bash $USERNAME
# Install awk separately with fault tolerance
# awk is a virtual package with multiple implementations (gawk, mawk, original-awk).
# Separated because TensorRT-LLM builds failed on awk package conflicts.
# This prevents main package installation failures due to awk availability issues.
RUN (apt-get install -y gawk || \
apt-get install -y mawk || \
apt-get install -y original-awk || \
echo "Warning: Could not install any awk implementation") && \
(which awk && echo "awk successfully installed: $(which awk)" || echo "awk not available")
# Add NVIDIA devtools repository and install development tools
RUN wget -qO - https://developer.download.nvidia.com/devtools/repos/ubuntu2404/${ARCH}/nvidia.pub | gpg --dearmor -o /etc/apt/keyrings/nvidia-devtools.gpg && \
echo "deb [signed-by=/etc/apt/keyrings/nvidia-devtools.gpg] https://developer.download.nvidia.com/devtools/repos/ubuntu2404/${ARCH} /" | tee /etc/apt/sources.list.d/nvidia-devtools.list && \
apt-get update && \
apt-get install -y nsight-systems-2025.5.1
# Clean up package lists at the end
RUN rm -rf /var/lib/apt/lists/*
# Set workspace directory variable
ENV WORKSPACE_DIR=${WORKSPACE_DIR}
# Development environment variables for the local-dev target
# Path configuration notes:
# - DYNAMO_HOME: Main project directory (workspace mount point)
# - CARGO_TARGET_DIR: Build artifacts in workspace/target for persistence
# - CARGO_HOME: Must be in $HOME/.cargo (not workspace) because:
# * Workspace gets mounted to different paths where cargo binaries may not exist
# * Contains critical cargo binaries and registry that need consistent paths
# - RUSTUP_HOME: Must be in $HOME/.rustup (not workspace) because:
# * Contains rust toolchain binaries that must be at expected system paths
# * Workspace mount point would break rustup's toolchain resolution
# - PATH: Includes cargo binaries for rust tool access
ENV DYNAMO_HOME=${WORKSPACE_DIR}
ENV CARGO_TARGET_DIR=${WORKSPACE_DIR}/target
ENV CARGO_HOME=${HOME}/.cargo
ENV RUSTUP_HOME=${HOME}/.rustup
ENV PATH=${CARGO_HOME}/bin:$PATH
# Copy Rust toolchain from system directories to user home directories with proper ownership
RUN rsync -a --chown=$USER_UID:$USER_GID /usr/local/rustup/ $RUSTUP_HOME/
RUN rsync -a --chown=$USER_UID:$USER_GID /usr/local/cargo/ $CARGO_HOME/
# Copy virtual environment with proper ownership using rsync instead of chown.
# Why rsync instead of chown -R:
# chown -R is extremely slow in Docker containers, especially on large directory trees
# like Python virtual environments with thousands of files. This is a well-documented
# Docker performance issue. rsync --chown is 3-4x faster as it sets ownership during copy.
RUN rsync -a --chown=$USER_UID:$USER_GID ${VIRTUAL_ENV}/ /tmp/venv-temp/ && \
rm -rf ${VIRTUAL_ENV} && \
mv /tmp/venv-temp ${VIRTUAL_ENV}
# At this point, we are executing as the ubuntu user
USER $USERNAME
ENV HOME=/home/$USERNAME
WORKDIR $HOME
# https://code.visualstudio.com/remote/advancedcontainers/persist-bash-history
RUN SNIPPET="export PROMPT_COMMAND='history -a' && export HISTFILE=$HOME/.commandhistory/.bash_history" \
&& mkdir -p $HOME/.commandhistory \
&& touch $HOME/.commandhistory/.bash_history \
&& echo "$SNIPPET" >> "$HOME/.bashrc"
RUN mkdir -p /home/$USERNAME/.cache/
ENTRYPOINT ["/opt/nvidia/nvidia_entrypoint.sh"]
CMD []
container/Dockerfile.vllm
View file @ 10bfb73a
...@@ -289,125 +289,6 @@ RUN --mount=type=bind,source=./container/launch_message.txt,target=/workspace/la ...@@ -289,125 +289,6 @@ RUN --mount=type=bind,source=./container/launch_message.txt,target=/workspace/la
ENTRYPOINT ["/opt/nvidia/nvidia_entrypoint.sh"] ENTRYPOINT ["/opt/nvidia/nvidia_entrypoint.sh"]
CMD [] CMD []
#######################################################################
########## Development (Dev Container only) ###########################
#######################################################################
#
# This stage is for Dev Container plug-in use only.
# It provides a local development environment with extra tools and dependencies
# not present in the base runtime image.
FROM runtime AS local-dev
# Don't want ubuntu to be editable, just change uid and gid.
ENV USERNAME=ubuntu
ARG USER_UID
ARG USER_GID
ARG WORKSPACE_DIR=/workspace
# Install utilities as root
RUN apt-get update -y && \
apt-get install -y --no-install-recommends \
# Install utilities
nvtop \
wget \
tmux \
vim \
git \
openssh-client \
iproute2 \
rsync \
zip \
unzip \
htop \
# Build Dependencies
autoconf \
automake \
cmake \
libtool \
meson \
net-tools \
pybind11-dev \
# Rust build dependencies
clang \
libclang-dev \
protobuf-compiler && \
rm -rf /var/lib/apt/lists/*
ARG ARCH
RUN wget -qO - https://developer.download.nvidia.com/devtools/repos/ubuntu2404/${ARCH}/nvidia.pub | gpg --dearmor -o /etc/apt/keyrings/nvidia-devtools.gpg && \
echo "deb [signed-by=/etc/apt/keyrings/nvidia-devtools.gpg] https://developer.download.nvidia.com/devtools/repos/ubuntu2404/${ARCH} /" | tee /etc/apt/sources.list.d/nvidia-devtools.list && \
apt-get update && \
apt-get install -y nsight-systems-2025.5.1 && \
rm -rf /var/lib/apt/lists/*
COPY --from=runtime /usr/local/bin /usr/local/bin
# https://code.visualstudio.com/remote/advancedcontainers/add-nonroot-user
# Will use the default ubuntu user, but give sudo access
# Needed so files permissions aren't set to root ownership when writing from inside container
RUN apt-get update && apt-get install -y sudo gnupg2 gnupg1 \
&& echo "$USERNAME ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/$USERNAME \
&& chmod 0440 /etc/sudoers.d/$USERNAME \
&& mkdir -p /home/$USERNAME \
&& groupmod -g $USER_GID $USERNAME \
&& usermod -u $USER_UID -g $USER_GID $USERNAME \
&& chown -R $USERNAME:$USERNAME /home/$USERNAME \
&& rm -rf /var/lib/apt/lists/* \
&& chsh -s /bin/bash $USERNAME
# At this point, we are executing as the ubuntu user
USER $USERNAME
ENV HOME=/home/$USERNAME
WORKDIR $HOME
# Set workspace directory variable
ENV WORKSPACE_DIR=${WORKSPACE_DIR}
# Development environment variables for the dev target
# Path configuration notes:
# - DYNAMO_HOME: Main project directory (workspace mount point)
# - CARGO_TARGET_DIR: Build artifacts in workspace/target for persistence
# - CARGO_HOME: Must be in $HOME/.cargo (not workspace) because:
# * Workspace gets mounted to different paths where cargo binaries may not exist
# * Contains critical cargo binaries and registry that need consistent paths
# - RUSTUP_HOME: Must be in $HOME/.rustup (not workspace) because:
# * Contains rust toolchain binaries that must be at expected system paths
# * Workspace mount point would break rustup's toolchain resolution
# - PATH: Includes cargo binaries for rust tool access
ENV DYNAMO_HOME=${WORKSPACE_DIR}
ENV CARGO_TARGET_DIR=${WORKSPACE_DIR}/target
ENV CARGO_HOME=${HOME}/.cargo
ENV RUSTUP_HOME=${HOME}/.rustup
ENV PATH=${CARGO_HOME}/bin:$PATH
COPY --from=dynamo_base --chown=$USER_UID:$USER_GID /usr/local/rustup $RUSTUP_HOME
COPY --from=dynamo_base --chown=$USER_UID:$USER_GID /usr/local/cargo $CARGO_HOME
# This is a slow operation (~40s on my cpu)
# Much better than chown -R $USERNAME:$USERNAME /opt/dynamo/venv (~10min on my cpu)
COPY --from=runtime --chown=$USER_UID:$USER_GID ${VIRTUAL_ENV} ${VIRTUAL_ENV}
# so we can use maturin develop
RUN uv pip install maturin[patchelf]
# Make sure to sync this with the one specified on README.md.
# This is a generic PYTHONPATH which works for all the frameworks, so some paths may not be relevant for this particular framework.
ENV PYTHONPATH=${WORKSPACE_DIR}/components/metrics/src:${WORKSPACE_DIR}/components/frontend/src:${WORKSPACE_DIR}/components/planner/src:${WORKSPACE_DIR}/components/backends/mocker/src:${WORKSPACE_DIR}/components/backends/trtllm/src:${WORKSPACE_DIR}/components/backends/vllm/src:${WORKSPACE_DIR}/components/backends/sglang/src:${WORKSPACE_DIR}/components/backends/llama_cpp/src
# https://code.visualstudio.com/remote/advancedcontainers/persist-bash-history
RUN SNIPPET="export PROMPT_COMMAND='history -a' && export HISTFILE=$HOME/.commandhistory/.bash_history" \
&& mkdir -p $HOME/.commandhistory \
&& touch $HOME/.commandhistory/.bash_history \
&& echo "$SNIPPET" >> "$HOME/.bashrc"
RUN mkdir -p /home/$USERNAME/.cache/
ENTRYPOINT ["/opt/nvidia/nvidia_entrypoint.sh"]
CMD []
########################################################### ###########################################################
########## Development (run.sh, runs as root user) ######## ########## Development (run.sh, runs as root user) ########
########################################################### ###########################################################
......
container/README.md
View file @ 10bfb73a
...@@ -38,8 +38,8 @@ The `build.sh` and `run.sh` scripts are convenience wrappers that simplify commo ...@@ -38,8 +38,8 @@ The `build.sh` and `run.sh` scripts are convenience wrappers that simplify commo
These targets are specified with `build.sh --target <target>` and correspond to Docker multi-stage build targets defined in the Dockerfiles (e.g., `FROM somebase AS <target>`). Some commonly used targets include: These targets are specified with `build.sh --target <target>` and correspond to Docker multi-stage build targets defined in the Dockerfiles (e.g., `FROM somebase AS <target>`). Some commonly used targets include:
- `runtime` - For running pre-built containers without development tools (minimal size) - `runtime` - For running pre-built containers without development tools (minimal size)
- `dev` - For development with full toolchain (git, vim, build tools, etc.) - `dev` - For development (inferencing/benchmarking/etc, runs as root user)
- `local-dev` - For development with user-based permissions matching host UID/GID - `local-dev` - For development with local user permissions matching host UID/GID. This is useful when mounting host partitions (with local user permissions) to Docker partitions.
Additional targets are available in the Dockerfiles for specific build stages and use cases. Additional targets are available in the Dockerfiles for specific build stages and use cases.
...@@ -73,24 +73,24 @@ Compatibility │ Legacy workflows, │ workspace writable on NFS│work ...@@ -73,24 +73,24 @@ Compatibility │ Legacy workflows, │ workspace writable on NFS│work
## Usage Guidelines ## Usage Guidelines
- **Use dev + `run.sh`**: `run.sh` script for command-line development by root user - **Use dev + `run.sh`**: for command-line testing. Runs as root user
- **Use local-dev + `run.sh`**: `run.sh` script for command-line development using your local user ID - **Use local-dev + `run.sh`**: for command-line development and Docker mounted partitions using your local user ID
- **Use local-dev + Dev Container**: VS Code/Cursor Dev Container Plugin, using your local user ID - **Use local-dev + Dev Container**: VS Code/Cursor Dev Container Plugin, using your local user ID
## Example Commands ## Example Commands
### 1. dev + `run.sh`: ### 1. dev + `run.sh` (runs as root):
```bash ```bash
run.sh --mount-workspace ... run.sh ...
``` ```
### 2. local-dev + `run.sh`: ### 2. local-dev + `run.sh` (runs as the local user):
```bash ```bash
run.sh --mount-workspace --image dynamo:latest-vllm-local-dev ... run.sh --mount-workspace --image dynamo:latest-vllm-local-dev ...
``` ```
### 3. local-dev + Dev Container: ### 3. local-dev + Dev Container Extension:
Use VS Code/Cursor Dev Container plugin with devcontainer.json configuration Use VS Code/Cursor Dev Container Extension with devcontainer.json configuration
## Build and Run Scripts Overview ## Build and Run Scripts Overview
...@@ -107,22 +107,22 @@ The `build.sh` script is responsible for building Docker images for different AI ...@@ -107,22 +107,22 @@ The `build.sh` script is responsible for building Docker images for different AI
**Key Features:** **Key Features:**
- **Framework Support**: vLLM (default when --framework not specified), TensorRT-LLM, SGLang, or NONE - **Framework Support**: vLLM (default when --framework not specified), TensorRT-LLM, SGLang, or NONE
- **Multi-stage Builds**: Build process with base images - **Multi-stage Builds**: Build process with base images
- **Development Targets**: Supports `dev` and `local-dev` targets - **Development Targets**: Supports `dev` target and `local-dev` target
- **Build Caching**: Docker layer caching and sccache support - **Build Caching**: Docker layer caching and sccache support
- **GPU Optimization**: CUDA, EFA, and NIXL support - **GPU Optimization**: CUDA, EFA, and NIXL support
**Common Usage Examples:** **Common Usage Examples:**
```bash ```bash
# Build vLLM image (default) # Build vLLM dev image called dynamo:latest-vllm (default). This runs as root and is fine to use for inferencing/benchmarking, etc.
./build.sh ./build.sh
# Build with specific framework # Build both development and local-dev images (integrated into build.sh). While the dev image runs as root, the local-dev image will run as the local user, which is useful when mounting partitions. It will also contain development tools.
./build.sh --framework trtllm
# Build local development image
./build.sh --framework vllm --target local-dev ./build.sh --framework vllm --target local-dev
# Build TensorRT-LLM development image called dynamo:latest-trtllm
./build.sh --framework trtllm
# Build with custom tag # Build with custom tag
./build.sh --framework sglang --tag my-custom-tag ./build.sh --framework sglang --tag my-custom-tag
...@@ -136,6 +136,23 @@ The `build.sh` script is responsible for building Docker images for different AI ...@@ -136,6 +136,23 @@ The `build.sh` script is responsible for building Docker images for different AI
./build.sh --build-arg CUSTOM_ARG=value ./build.sh --build-arg CUSTOM_ARG=value
``` ```
### build.sh --dev-image - Local Development Image Builder
The `build.sh --dev-image` option takes a dev image and then builds a local-dev image, which contains proper local user permissions. It also includes extra developer utilities (debugging tools, text editors, system monitors, etc.).
**Common Usage Examples:**
```bash
# Build local-dev image from dev image dynamo:latest-vllm
./build.sh --dev-image dynamo:latest-vllm --framework vllm
# Build with custom tag from dev image dynamo:latest-vllm
./build.sh --dev-image dynamo:latest-vllm --framework vllm --tag my-local:dev
# Dry run to see what would be built
./build.sh --dev-image dynamo:latest-vllm --framework vllm --dry-run
```
### run.sh - Container Runtime Manager ### run.sh - Container Runtime Manager
The `run.sh` script launches Docker containers with the appropriate configuration for development and inference workloads. The `run.sh` script launches Docker containers with the appropriate configuration for development and inference workloads.
...@@ -156,42 +173,39 @@ The `run.sh` script launches Docker containers with the appropriate configuratio ...@@ -156,42 +173,39 @@ The `run.sh` script launches Docker containers with the appropriate configuratio
**Common Usage Examples:** **Common Usage Examples:**
```bash ```bash
# Basic container launch # Basic container launch (inference/production)
./run.sh ./run.sh --image dynamo:latest-vllm
# Mount workspace for development # Mount workspace for development (use local-dev image for local user permissions)
./run.sh --mount-workspace ./run.sh --image dynamo:latest-vllm-local-dev --mount-workspace
# Use specific image and framework # Use specific image and framework for development
./run.sh --image dynamo:latest-vllm --framework vllm ./run.sh --image v0.1.0.dev.08cc44965-vllm-local-dev --framework vllm --mount-workspace
# Interactive shell with workspace mounted # Interactive development shell with workspace mounted
./run.sh --mount-workspace -it -- bash ./run.sh --image dynamo:latest-vllm-local-dev --mount-workspace -it -- bash
# Run with custom environment variables # Development with custom environment variables
./run.sh -e CUDA_VISIBLE_DEVICES=0,1 --mount-workspace ./run.sh --image dynamo:latest-vllm-local-dev -e CUDA_VISIBLE_DEVICES=0,1 --mount-workspace
# Run without GPU access # Production inference without GPU access
./run.sh --gpus none ./run.sh --image dynamo:latest-vllm --gpus none
# Dry run to see docker command # Dry run to see docker command
./run.sh --dry-run ./run.sh --dry-run
# Run with custom volume mounts # Development with custom volume mounts
./run.sh -v /host/path:/container/path --mount-workspace ./run.sh --image dynamo:latest-vllm-local-dev -v /host/path:/container/path --mount-workspace
# Launch with specific container name
./run.sh --name my-dynamo-container --mount-workspace
``` ```
## Workflow Examples ## Workflow Examples
### Development Workflow ### Development Workflow
```bash ```bash
# 1. Build development image # 1. Build local-dev image (creates both dynamo:latest-vllm and dynamo:latest-vllm-local-dev)
./build.sh --framework vllm --target local-dev ./build.sh --framework vllm --target local-dev
# 2. Run development container # 2. Run development container using the local-dev image
./run.sh --image dynamo:latest-vllm-local-dev --mount-workspace -it ./run.sh --image dynamo:latest-vllm-local-dev --mount-workspace -it
# 3. Inside container, run inference (requires both frontend and backend) # 3. Inside container, run inference (requires both frontend and backend)
...@@ -208,7 +222,7 @@ python -m dynamo.vllm --model Qwen/Qwen3-0.6B --gpu-memory-utilization 0.50 & ...@@ -208,7 +222,7 @@ python -m dynamo.vllm --model Qwen/Qwen3-0.6B --gpu-memory-utilization 0.50 &
./build.sh --framework vllm --release-build ./build.sh --framework vllm --release-build
# 2. Run production container # 2. Run production container
./run.sh --image dynamo:latest-vllm --gpus all ./run.sh --image dynamo:latest-vllm-local-dev --gpus all
``` ```
### Testing Workflow ### Testing Workflow
...@@ -216,6 +230,6 @@ python -m dynamo.vllm --model Qwen/Qwen3-0.6B --gpu-memory-utilization 0.50 & ...@@ -216,6 +230,6 @@ python -m dynamo.vllm --model Qwen/Qwen3-0.6B --gpu-memory-utilization 0.50 &
# 1. Build with no cache for clean build # 1. Build with no cache for clean build
./build.sh --framework vllm --no-cache ./build.sh --framework vllm --no-cache
# 2. Test container functionality # 2. Test container functionality (--image defaults to dynamo:latest-vllm)
./run.sh --mount-workspace -it -- python -m pytest tests/ ./run.sh --mount-workspace -it -- python -m pytest tests/
``` ```
container/build.sh
View file @ 10bfb73a
...@@ -204,6 +204,7 @@ get_options() { ...@@ -204,6 +204,7 @@ get_options() {
fi fi
;; ;;
--base-image) --base-image)
# Note: --base-image cannot be used with --dev-image
if [ "$2" ]; then if [ "$2" ]; then
BASE_IMAGE=$2 BASE_IMAGE=$2
shift shift
...@@ -227,6 +228,30 @@ get_options() { ...@@ -227,6 +228,30 @@ get_options() {
missing_requirement "$1" missing_requirement "$1"
fi fi
;; ;;
--dev-image)
if [ "$2" ]; then
DEV_IMAGE_INPUT=$2
shift
else
missing_requirement "$1"
fi
;;
--uid)
if [ "$2" ]; then
CUSTOM_UID=$2
shift
else
missing_requirement "$1"
fi
;;
--gid)
if [ "$2" ]; then
CUSTOM_GID=$2
shift
else
missing_requirement "$1"
fi
;;
--build-arg) --build-arg)
if [ "$2" ]; then if [ "$2" ]; then
BUILD_ARGS+="--build-arg $2 " BUILD_ARGS+="--build-arg $2 "
...@@ -320,6 +345,23 @@ get_options() { ...@@ -320,6 +345,23 @@ get_options() {
shift shift
done done
# Validate argument combinations
if [[ -n "${DEV_IMAGE_INPUT:-}" && -n "${BASE_IMAGE:-}" ]]; then
error "ERROR: --dev-image cannot be used with --base-image. Use --dev-image to build from existing images or --base-image to build new images."
fi
# Validate that --target and --dev-image cannot be used together
if [[ -n "${DEV_IMAGE_INPUT:-}" && -n "${TARGET:-}" ]]; then
error "ERROR: --target cannot be used with --dev-image. Use --target to build from scratch or --dev-image to build from existing images."
fi
# Validate that --uid and --gid are only used with local-dev related options
if [[ -n "${CUSTOM_UID:-}" || -n "${CUSTOM_GID:-}" ]]; then
if [[ -z "${DEV_IMAGE_INPUT:-}" && "${TARGET:-}" != "local-dev" ]]; then
error "ERROR: --uid and --gid can only be used with --dev-image or --target local-dev"
fi
fi
if [ -z "$FRAMEWORK" ]; then if [ -z "$FRAMEWORK" ]; then
FRAMEWORK=$DEFAULT_FRAMEWORK FRAMEWORK=$DEFAULT_FRAMEWORK
fi fi
...@@ -352,7 +394,7 @@ get_options() { ...@@ -352,7 +394,7 @@ get_options() {
if [ -z "$TAG" ]; then if [ -z "$TAG" ]; then
TAG="--tag dynamo:${VERSION}-${FRAMEWORK,,}" TAG="--tag dynamo:${VERSION}-${FRAMEWORK,,}"
if [ -n "${TARGET}" ]; then if [ -n "${TARGET}" ] && [ "${TARGET}" != "local-dev" ]; then
TAG="${TAG}-${TARGET}" TAG="${TAG}-${TARGET}"
fi fi
fi fi
...@@ -419,6 +461,9 @@ show_help() { ...@@ -419,6 +461,9 @@ show_help() {
echo " [--cache-from cache location to start from]" echo " [--cache-from cache location to start from]"
echo " [--cache-to location where to cache the build output]" echo " [--cache-to location where to cache the build output]"
echo " [--tag tag for image]" echo " [--tag tag for image]"
echo " [--dev-image dev image to build local-dev from]"
echo " [--uid user ID for local-dev images (only with --dev-image or --target local-dev)]"
echo " [--gid group ID for local-dev images (only with --dev-image or --target local-dev)]"
echo " [--no-cache disable docker build cache]" echo " [--no-cache disable docker build cache]"
echo " [--dry-run print docker commands without running]" echo " [--dry-run print docker commands without running]"
echo " [--build-context name=path to add build context]" echo " [--build-context name=path to add build context]"
...@@ -468,8 +513,73 @@ fi ...@@ -468,8 +513,73 @@ fi
# Add NIXL_REF as a build argument # Add NIXL_REF as a build argument
BUILD_ARGS+=" --build-arg NIXL_REF=${NIXL_REF} " BUILD_ARGS+=" --build-arg NIXL_REF=${NIXL_REF} "
# Function to build local-dev image with header
build_local_dev_with_header() {
local dev_base_image="$1"
local tags="$2"
local success_msg="$3"
local header_title="$4"
echo "======================================"
echo "$header_title"
echo "======================================"
# Get user info right before using it
USER_UID=${CUSTOM_UID:-$(id -u)}
USER_GID=${CUSTOM_GID:-$(id -g)}
# Set up dockerfile path
DOCKERFILE_LOCAL_DEV="${SOURCE_DIR}/Dockerfile.local_dev"
if [[ ! -f "$DOCKERFILE_LOCAL_DEV" ]]; then
echo "ERROR: Dockerfile.local_dev not found at: $DOCKERFILE_LOCAL_DEV"
exit 1
fi
echo "Building new local-dev image from: $dev_base_image"
echo "User 'ubuntu' will have UID: $USER_UID, GID: $USER_GID"
# Show the docker command being executed if not in dry-run mode
if [ -z "$RUN_PREFIX" ]; then
set -x
fi
$RUN_PREFIX docker build \
--build-arg DEV_BASE="$dev_base_image" \
--build-arg USER_UID="$USER_UID" \
--build-arg USER_GID="$USER_GID" \
--build-arg ARCH="$ARCH" \
--file "$DOCKERFILE_LOCAL_DEV" \
$tags \
"$SOURCE_DIR" || {
{ set +x; } 2>/dev/null
echo "ERROR: Failed to build local_dev image"
exit 1
}
{ set +x; } 2>/dev/null
echo "$success_msg"
# Show usage instructions
echo ""
echo "To run the local-dev image as the local user ($USER_UID/$USER_GID):"
# Extract the last tag from the tags string
last_tag=$(echo "$tags" | grep -o -- '--tag [^ ]*' | tail -1 | cut -d' ' -f2)
# Calculate relative path to run.sh from current working directory
# Get the directory where build.sh is located
build_dir="$(dirname "${BASH_SOURCE[0]}")"
# Get the absolute path to run.sh (in the same directory as build.sh)
run_abs_path="$(realpath "$build_dir/run.sh")"
# Calculate relative path from current PWD to run.sh
run_path="$(python3 -c "import os; print(os.path.relpath('$run_abs_path', '$PWD'))")"
echo " $run_path --image $last_tag --mount-workspace ..."
}
# Handle local-dev target
if [[ $TARGET == "local-dev" ]]; then if [[ $TARGET == "local-dev" ]]; then
BUILD_ARGS+=" --build-arg USER_UID=$(id -u) --build-arg USER_GID=$(id -g) " LOCAL_DEV_BUILD=true
TARGET_STR="--target dev"
fi fi
# BUILD DEV IMAGE # BUILD DEV IMAGE
...@@ -607,7 +717,7 @@ if [ "$USE_SCCACHE" = true ]; then ...@@ -607,7 +717,7 @@ if [ "$USE_SCCACHE" = true ]; then
fi fi
LATEST_TAG="--tag dynamo:latest-${FRAMEWORK,,}" LATEST_TAG="--tag dynamo:latest-${FRAMEWORK,,}"
if [ -n "${TARGET}" ]; then if [ -n "${TARGET}" ] && [ "${TARGET}" != "local-dev" ]; then
LATEST_TAG="${LATEST_TAG}-${TARGET}" LATEST_TAG="${LATEST_TAG}-${TARGET}"
fi fi
...@@ -617,24 +727,69 @@ if [ -z "$RUN_PREFIX" ]; then ...@@ -617,24 +727,69 @@ if [ -z "$RUN_PREFIX" ]; then
set -x set -x
fi fi
# TODO: Follow 2-step build process for all frameworks once necessary changes are made to the sglang and TRT-LLM backend Dockerfiles.
if [[ $FRAMEWORK == "VLLM" ]] || [[ $FRAMEWORK == "SGLANG" ]]; then # Skip Build 1 and Build 2 if DEV_IMAGE_INPUT is set (we'll handle it at the bottom)
# Define base image tag before using it if [[ -z "${DEV_IMAGE_INPUT:-}" ]]; then
DYNAMO_BASE_IMAGE="dynamo-base:${VERSION}" # TODO: Follow 2-step build process for all frameworks once necessary changes are made to the sglang and TRT-LLM backend Dockerfiles.
# Start base image build if [[ $FRAMEWORK == "VLLM" ]] || [[ $FRAMEWORK == "SGLANG" ]]; then
echo "======================================" # Define base image tag before using it
echo "Starting Build 1: Base Image" DYNAMO_BASE_IMAGE="dynamo-base:${VERSION}"
echo "======================================" # Start base image build
$RUN_PREFIX docker build -f "${SOURCE_DIR}/Dockerfile" --target dev $PLATFORM $BUILD_ARGS $CACHE_FROM $CACHE_TO --tag $DYNAMO_BASE_IMAGE $BUILD_CONTEXT_ARG $BUILD_CONTEXT $NO_CACHE echo "======================================"
# Start framework build echo "Starting Build 1: Base Image"
echo "======================================" echo "======================================"
echo "Starting Build 2: Framework Image" $RUN_PREFIX docker build -f "${SOURCE_DIR}/Dockerfile" --target dev $PLATFORM $BUILD_ARGS $CACHE_FROM $CACHE_TO --tag $DYNAMO_BASE_IMAGE $BUILD_CONTEXT_ARG $BUILD_CONTEXT $NO_CACHE
echo "======================================" # Start framework build
BUILD_ARGS+=" --build-arg DYNAMO_BASE_IMAGE=${DYNAMO_BASE_IMAGE}" echo "======================================"
$RUN_PREFIX docker build -f $DOCKERFILE $TARGET_STR $PLATFORM $BUILD_ARGS $CACHE_FROM $CACHE_TO $TAG $LATEST_TAG $BUILD_CONTEXT_ARG $BUILD_CONTEXT $NO_CACHE echo "Starting Build 2: Framework Image"
else echo "======================================"
$RUN_PREFIX docker build -f $DOCKERFILE $TARGET_STR $PLATFORM $BUILD_ARGS $CACHE_FROM $CACHE_TO $TAG $LATEST_TAG $BUILD_CONTEXT_ARG $BUILD_CONTEXT $NO_CACHE BUILD_ARGS+=" --build-arg DYNAMO_BASE_IMAGE=${DYNAMO_BASE_IMAGE}"
$RUN_PREFIX docker build -f $DOCKERFILE $TARGET_STR $PLATFORM $BUILD_ARGS $CACHE_FROM $CACHE_TO $TAG $LATEST_TAG $BUILD_CONTEXT_ARG $BUILD_CONTEXT $NO_CACHE
else
$RUN_PREFIX docker build -f $DOCKERFILE $TARGET_STR $PLATFORM $BUILD_ARGS $CACHE_FROM $CACHE_TO $TAG $LATEST_TAG $BUILD_CONTEXT_ARG $BUILD_CONTEXT $NO_CACHE
fi
fi fi
# Handle --dev-image option (build local-dev from existing dev image)
if [[ -n "${DEV_IMAGE_INPUT:-}" ]]; then
# Validate that the dev image is not already a local-dev image
if [[ "$DEV_IMAGE_INPUT" == *"-local-dev" ]]; then
echo "ERROR: Cannot use local-dev image as dev image input: '$DEV_IMAGE_INPUT'"
exit 1
fi
# Build tag arguments - always add -local-dev suffix for --dev-image
# Generate local-dev tag from input image
if [[ "$DEV_IMAGE_INPUT" == *:* ]]; then
LOCAL_DEV_TAG="--tag ${DEV_IMAGE_INPUT}-local-dev"
else
LOCAL_DEV_TAG="--tag ${DEV_IMAGE_INPUT}:latest-local-dev"
fi
build_local_dev_with_header "$DEV_IMAGE_INPUT" "$LOCAL_DEV_TAG" "Successfully built local-dev image: ${LOCAL_DEV_TAG#--tag }" "Building Local-Dev Image"
elif [[ "${LOCAL_DEV_BUILD:-}" == "true" ]]; then
# Use the first tag name (TAG) if available, otherwise use latest
if [[ -n "$TAG" ]]; then
DEV_IMAGE=$(echo "$TAG" | sed 's/--tag //' | sed 's/-local-dev$//')
else
DEV_IMAGE="dynamo:latest-${FRAMEWORK,,}"
fi
# Build local-dev tags from existing tags
LOCAL_DEV_TAGS=""
if [[ -n "$TAG" ]]; then
# Extract tag name, remove any existing -local-dev suffix, then add -local-dev
TAG_NAME=$(echo "$TAG" | sed 's/--tag //' | sed 's/-local-dev$//')
LOCAL_DEV_TAGS+=" --tag ${TAG_NAME}-local-dev"
fi
if [[ -n "$LATEST_TAG" ]]; then
# Extract tag name, remove any existing -local-dev suffix, then add -local-dev
LATEST_TAG_NAME=$(echo "$LATEST_TAG" | sed 's/--tag //' | sed 's/-local-dev$//')
LOCAL_DEV_TAGS+=" --tag ${LATEST_TAG_NAME}-local-dev"
fi
build_local_dev_with_header "$DEV_IMAGE" "$LOCAL_DEV_TAGS" "Successfully built local-dev images" "Starting Build 3: Local-Dev Image"
fi
{ set +x; } 2>/dev/null { set +x; } 2>/dev/null
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment