refactor: update Dockerfiles to use group writes for development, saving build time (#4657)

Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com> Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>

refactor: update Dockerfiles to use group writes for development, saving build time (#4657)
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com> Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
0dd51694 · Keiven C · GitHub · 80dfb82c · 0dd51694 · 0dd51694
Unverified Commit 0dd51694 authored Dec 08, 2025 by Keiven C Committed by GitHub Dec 08, 2025
5 changed files
--- a/container/Dockerfile.local_dev
+++ b/container/Dockerfile.local_dev
@@ -14,22 +14,16 @@
 ARG DEV_BASE=""
 FROM ${DEV_BASE} AS local-dev
-# Don't want dynamo to be editable, just change uid and gid.
+# Switch to root for package installation (dev stage ends as dynamo user)
-ENV USERNAME=dynamo
+USER root
-ARG USER_UID
+# Reset SHELL to non-login bash (dev stage uses login shell)
-ARG USER_GID
+SHELL ["/bin/bash", "-c"]
-ARG WORKSPACE_DIR=/workspace
-ARG DYNAMO_COMMIT_SHA
-ENV DYNAMO_COMMIT_SHA=$DYNAMO_COMMIT_SHA
-ARG ARCH
 # Update package lists and install developer utilities. Some of these may exist in the base image,
 # but to ensure consistency across all dev images, we explicitly list all required dev tools here.
 RUN apt-get update && apt-get install -y \
    # Development utilities
-    curl wget git vim nano \
+    curl wget git vim nano less \
    # System utilities
    htop nvtop tmux screen \
    # Network utilities
@@ -47,20 +41,6 @@ RUN apt-get update && apt-get install -y \
    # Shell utilities
    zsh fish bash-completion
-# https://code.visualstudio.com/remote/advancedcontainers/add-nonroot-user
-# Configure user with sudo access for Dev Container workflows
-RUN apt-get install -y sudo gnupg2 gnupg1 \
-    && echo "$USERNAME ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/$USERNAME \
-    && chmod 0440 /etc/sudoers.d/$USERNAME \
-    && mkdir -p /home/$USERNAME \
-    # Handle GID conflicts: if target GID exists and it's not our group, remove it
-    && (getent group $USER_GID | grep -v "^$USERNAME:" && groupdel $(getent group $USER_GID | cut -d: -f1) || true) \
-    # Create group if it doesn't exist, otherwise modify existing group
-    && (getent group $USERNAME > /dev/null 2>&1 && groupmod -g $USER_GID $USERNAME || groupadd -g $USER_GID $USERNAME) \
-    && usermod -u $USER_UID -g $USER_GID $USERNAME \
-    && chown -R $USERNAME:$USERNAME /home/$USERNAME \
-    && chsh -s /bin/bash $USERNAME
 # Install awk separately with fault tolerance
 # awk is a virtual package with multiple implementations (gawk, mawk, original-awk).
 # Separated because TensorRT-LLM builds failed on awk package conflicts.
@@ -71,12 +51,35 @@ RUN (apt-get install -y gawk || \
     echo "Warning: Could not install any awk implementation") && \
    (which awk && echo "awk successfully installed: $(which awk)" || echo "awk not available")
+# Don't want dynamo to be editable, just change uid and gid.
+ENV USERNAME=dynamo
+ARG USER_UID
+ARG USER_GID
+ARG WORKSPACE_DIR=/workspace
+ARG ARCH=amd64
 # Add NVIDIA devtools repository and install development tools
 RUN wget -qO - https://developer.download.nvidia.com/devtools/repos/ubuntu2404/${ARCH}/nvidia.pub | gpg --dearmor -o /etc/apt/keyrings/nvidia-devtools.gpg && \
    echo "deb [signed-by=/etc/apt/keyrings/nvidia-devtools.gpg] https://developer.download.nvidia.com/devtools/repos/ubuntu2404/${ARCH} /" | tee /etc/apt/sources.list.d/nvidia-devtools.list && \
    apt-get update && \
    apt-get install -y nsight-systems-2025.5.1
+# https://code.visualstudio.com/remote/advancedcontainers/add-nonroot-user
+# Configure user with sudo access for Dev Container workflows
+RUN apt-get install -y sudo gnupg2 gnupg1 \
+    && echo "$USERNAME ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/$USERNAME \
+    && chmod 0440 /etc/sudoers.d/$USERNAME \
+    && mkdir -p /home/$USERNAME \
+    # Handle GID conflicts: if target GID exists and it's not our group, remove it
+    && (getent group $USER_GID | grep -v "^$USERNAME:" && groupdel $(getent group $USER_GID | cut -d: -f1) || true) \
+    # Create group if it doesn't exist, otherwise modify existing group
+    && (getent group $USERNAME > /dev/null 2>&1 && groupmod -g $USER_GID $USERNAME || groupadd -g $USER_GID $USERNAME) \
+    && usermod -u $USER_UID -g $USER_GID -G 0 $USERNAME \
+    && chown -R $USERNAME:$USERNAME /home/$USERNAME \
+    && chsh -s /bin/bash $USERNAME
 # Clean up package lists at the end
 RUN rm -rf /var/lib/apt/lists/*
@@ -87,45 +90,26 @@ ENV WORKSPACE_DIR=${WORKSPACE_DIR}
 # Path configuration notes:
 # - DYNAMO_HOME: Main project directory (workspace mount point)
 # - CARGO_TARGET_DIR: Build artifacts in workspace/target for persistence
-# - CARGO_HOME: Must be in $HOME/.cargo (not workspace) because:
-#               * Workspace gets mounted to different paths where cargo binaries may not exist
-#               * Contains critical cargo binaries and registry that need consistent paths
-# - RUSTUP_HOME: Must be in $HOME/.rustup (not workspace) because:
-#                * Contains rust toolchain binaries that must be at expected system paths
-#                * Workspace mount point would break rustup's toolchain resolution
 # - PATH: Includes cargo binaries for rust tool access
 ENV HOME=/home/$USERNAME
 ENV DYNAMO_HOME=${WORKSPACE_DIR}
 ENV CARGO_TARGET_DIR=${WORKSPACE_DIR}/target
-ENV CARGO_HOME=${HOME}/.cargo
+# NOTE: CARGO_HOME and RUSTUP_HOME are already inherited from dev stage (Dockerfile.sglang|trtllm|vllm)
-ENV RUSTUP_HOME=${HOME}/.rustup
 ENV PATH=${CARGO_HOME}/bin:$PATH
-# Copy Rust toolchain from system directories to user home directories with proper ownership
+# Switch to dynamo user (dev stage has umask 002, so files should already be group-writable)
-RUN rsync -a --chown=$USER_UID:$USER_GID /usr/local/rustup/ $RUSTUP_HOME/
-RUN rsync -a --chown=$USER_UID:$USER_GID /usr/local/cargo/ $CARGO_HOME/
-# Copy virtual environment with proper ownership using rsync instead of chown.
-# Why rsync instead of chown -R:
-# chown -R is extremely slow in Docker containers, especially on large directory trees
-# like Python virtual environments with thousands of files. This is a well-documented
-# Docker performance issue. rsync --chown is 3-4x faster as it sets ownership during copy.
-RUN rsync -a --chown=$USER_UID:$USER_GID ${VIRTUAL_ENV}/ /tmp/venv-temp/ && \
-    rm -rf ${VIRTUAL_ENV} && \
-    mv /tmp/venv-temp ${VIRTUAL_ENV}
-# At this point, we are executing as the ubuntu user
 USER $USERNAME
 WORKDIR $HOME
 # https://code.visualstudio.com/remote/advancedcontainers/persist-bash-history
 RUN SNIPPET="export PROMPT_COMMAND='history -a' && export HISTFILE=$HOME/.commandhistory/.bash_history" \
    && mkdir -p $HOME/.commandhistory \
+    && chmod g+w $HOME/.commandhistory \
    && touch $HOME/.commandhistory/.bash_history \
    && echo "$SNIPPET" >> "$HOME/.bashrc"
-RUN mkdir -p /home/$USERNAME/.cache/
+RUN mkdir -p /home/$USERNAME/.cache/ \
+    && chmod g+w /home/$USERNAME/.cache/
 ENTRYPOINT ["/opt/nvidia/nvidia_entrypoint.sh"]
 CMD []
--- a/container/Dockerfile.sglang
+++ b/container/Dockerfile.sglang
 # syntax=docker/dockerfile:1.10.0
 # SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 # SPDX-License-Identifier: Apache-2.0
+#
+# Throughout this file, we make certain paths group-writable because this allows
+# both the dynamo user (UID 1000) and Dev Container users (UID != 1000) to work
+# properly without needing slow chown -R operations (which can add 2-10 extra
+# minutes).
+#
+# DEVELOPMENT PATHS THAT MUST BE GROUP-WRITABLE (for non-virtualenv containers):
+#   /workspace            - Users create/modify project files
+#   /home/dynamo          - Users create config/cache files
+#   /home/dynamo/.local   - SGLang uses $HOME/.local/lib/python3.10/site-packages for pip install
+#
+# HOW TO ACHIEVE GROUP-WRITABLE PERMISSIONS:
+# 1. SHELL + /etc/profile.d - Login shell sources umask 002 globally for all RUN commands (775/664)
+# 2. COPY --chmod=775       - Sets permissions on copied children (not destination)
+# 3. chmod g+w (no -R)      - Fixes destination dirs only (milliseconds vs minutes)
 # This section contains build arguments that are common and shared with
 # the plain Dockerfile, so they should NOT have a default. The source of truth is from build.sh.
@@ -190,18 +205,25 @@ RUN git clone --depth 1 --branch ${GDRCOPY_COMMIT} https://github.com/NVIDIA/gdr
 # Fix DeepEP IBGDA symlink
 RUN ln -sf /usr/lib/$(uname -m)-linux-gnu/libmlx5.so.1 /usr/lib/$(uname -m)-linux-gnu/libmlx5.so
-# Create dynamo user EARLY - before copying files, with group 0 for OpenShift compatibility
+# Create dynamo user with group 0 for OpenShift compatibility
 RUN userdel -r ubuntu > /dev/null 2>&1 || true \
    && useradd -m -s /bin/bash -g 0 dynamo \
    && [ `id -u dynamo` -eq 1000 ] \
    && mkdir -p /workspace /home/dynamo/.cache /opt/dynamo \
-    && chown -R dynamo: /sgl-workspace /workspace /home/dynamo /opt/dynamo \
+    # Non-recursive chown - only the directories themselves, not contents
-    && chmod -R g+w /sgl-workspace /workspace /home/dynamo/.cache /opt/dynamo
+    && chown dynamo:0 /sgl-workspace /workspace /home/dynamo /home/dynamo/.cache /opt/dynamo \
+    # No chmod needed: umask 002 handles new files, COPY --chmod handles copied content
+    # Set umask globally for all subsequent RUN commands (must be done as root before USER dynamo)
+    # NOTE: Setting ENV UMASK=002 does NOT work - umask is a shell builtin, not an environment variable
+    && mkdir -p /etc/profile.d && echo 'umask 002' > /etc/profile.d/00-umask.sh
 USER dynamo
 ENV HOME=/home/dynamo
+# This picks up the umask 002 from the /etc/profile.d/00-umask.sh file for subsequent RUN commands
+SHELL ["/bin/bash", "-l", "-o", "pipefail", "-c"]
-# Install SGLang (requires CUDA 12.8.1 or 12.9.1)
+# Install SGLang (requires CUDA 12.8.1 or 12.9.1). Note that when system-wide packages is not writable,
+# so it gets installed to ~/.local/lib/python<version>/site-packages.
 RUN python3 -m pip install --no-cache-dir --ignore-installed pip==25.3 setuptools==80.9.0 wheel==0.45.1 html5lib==1.1 six==1.17.0 \
    && git clone --depth 1 --branch v${SGLANG_COMMIT} https://github.com/sgl-project/sglang.git \
    && cd sglang \
@@ -279,8 +301,10 @@ RUN --mount=type=secret,id=aws-key-id,env=AWS_ACCESS_KEY_ID \
    NVSHMEM_DIR=${NVSHMEM_DIR} TORCH_CUDA_ARCH_LIST="9.0;10.0" pip install --no-build-isolation .
 # Copy rust installation from dynamo_base to avoid duplication efforts
-COPY --from=dynamo_base /usr/local/rustup /usr/local/rustup
+# Pattern: COPY --chmod=775 <path>; RUN chmod g+w <path> because COPY --chmod only affects <path>/*, not <path>
-COPY --from=dynamo_base /usr/local/cargo /usr/local/cargo
+COPY --from=dynamo_base --chown=dynamo:0 --chmod=775 /usr/local/rustup /usr/local/rustup
+COPY --from=dynamo_base --chown=dynamo:0 --chmod=775 /usr/local/cargo /usr/local/cargo
+RUN chmod g+w /usr/local/rustup /usr/local/cargo
 ENV RUSTUP_HOME=/usr/local/rustup \
    CARGO_HOME=/usr/local/cargo \
@@ -341,8 +365,9 @@ COPY --from=dynamo_base $NIXL_PREFIX $NIXL_PREFIX
 ENV PATH=/usr/local/bin/etcd/:/usr/local/cuda/nvvm/bin:${HOME}/.local/bin:$PATH
 # Install Dynamo wheels from dynamo_base wheelhouse
-COPY --chown=dynamo: benchmarks/ /opt/dynamo/benchmarks/
+# Pattern: COPY --chmod=775 <path>; chmod g+w <path> done later as root because COPY --chmod only affects <path>/*, not <path>
-COPY --chown=dynamo: --from=dynamo_base /opt/dynamo/wheelhouse/ /opt/dynamo/wheelhouse/
+COPY --chmod=775 --chown=dynamo:0 benchmarks/ /opt/dynamo/benchmarks/
+COPY --chmod=775 --chown=dynamo:0 --from=dynamo_base /opt/dynamo/wheelhouse/ /opt/dynamo/wheelhouse/
 RUN python3 -m pip install \
    /opt/dynamo/wheelhouse/ai_dynamo_runtime*.whl \
    /opt/dynamo/wheelhouse/ai_dynamo*any.whl \
@@ -361,7 +386,17 @@ RUN --mount=type=bind,source=./container/deps/requirements.txt,target=/tmp/requi
        --requirement /tmp/requirements.test.txt
 ## Copy attribution files and launch banner with correct ownership
-COPY --chown=dynamo: ATTRIBUTION* LICENSE /workspace/
+COPY --chmod=664 --chown=dynamo:0 ATTRIBUTION* LICENSE /workspace/
+# Copy tests, benchmarks, deploy and components for CI with correct ownership
+# Pattern: COPY --chmod=775 <path>; chmod g+w <path> done later as root because COPY --chmod only affects <path>/*, not <path>
+COPY --chmod=775 --chown=dynamo:0 pyproject.toml /workspace/
+COPY --chmod=775 --chown=dynamo:0 tests /workspace/tests
+COPY --chmod=775 --chown=dynamo:0 examples /workspace/examples
+COPY --chmod=775 --chown=dynamo:0 benchmarks /workspace/benchmarks
+COPY --chmod=775 --chown=dynamo:0 deploy /workspace/deploy
+COPY --chmod=775 --chown=dynamo:0 components/ /workspace/components/
+COPY --chmod=775 --chown=dynamo:0 recipes/ /workspace/recipes/
 # Setup launch banner in common directory accessible to all users
 RUN --mount=type=bind,source=./container/launch_message/runtime.txt,target=/opt/dynamo/launch_message.txt \
@@ -369,21 +404,16 @@ RUN --mount=type=bind,source=./container/launch_message/runtime.txt,target=/opt/
 # Setup environment for all users
 USER root
-RUN chmod 755 /opt/dynamo/.launch_screen && \
+# Fix directory permissions: COPY --chmod only affects contents, not the directory itself
+RUN chmod g+w /workspace /workspace/* /opt/dynamo /opt/dynamo/* && \
+    chown dynamo:0 /workspace /opt/dynamo/ && \
+    chmod 755 /opt/dynamo/.launch_screen && \
    echo 'source /opt/dynamo/venv/bin/activate' >> /etc/bash.bashrc && \
    echo 'cat /opt/dynamo/.launch_screen' >> /etc/bash.bashrc
 USER dynamo
 # Copy tests, benchmarks, deploy and components for CI with correct ownership
-COPY --chown=dynamo: pyproject.toml /workspace/
-COPY --chown=dynamo: tests /workspace/tests
-COPY --chown=dynamo: examples /workspace/examples
-COPY --chown=dynamo: benchmarks /workspace/benchmarks
-COPY --chown=dynamo: deploy /workspace/deploy
-COPY --chown=dynamo: components/ /workspace/components/
-COPY --chown=dynamo: recipes/ /workspace/recipes/
 ARG DYNAMO_COMMIT_SHA
 ENV DYNAMO_COMMIT_SHA=$DYNAMO_COMMIT_SHA
@@ -420,6 +450,8 @@ ENV VIRTUAL_ENV=/opt/dynamo/venv \
    PATH="/opt/dynamo/venv/bin:${PATH}"
 USER root
+# venv permissions are handled by umask 002 set earlier
 # Install development tools and utilities
 RUN apt-get update -y && \
    apt-get install -y --no-install-recommends  \
@@ -478,7 +510,7 @@ RUN curl --retry 3 --retry-delay 2 -LSso /usr/local/bin/clang-format https://git
    && rm -rf clangd_18.1.3 clangd.zip
 # Editable install of dynamo
-COPY README.md hatch_build.py /workspace/
+COPY --chmod=664 pyproject.toml README.md hatch_build.py /workspace/
 RUN python3 -m pip install --no-deps -e .
 # Install Python development packages
@@ -496,4 +528,4 @@ RUN python3 -m pip install --no-cache-dir \
    tabulate
 ENTRYPOINT ["/opt/nvidia/nvidia_entrypoint.sh"]
 CMD []
\ No newline at end of file
--- a/container/Dockerfile.trtllm
+++ b/container/Dockerfile.trtllm
 # SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 # SPDX-License-Identifier: Apache-2.0
+#
+# Throughout this file, we make certain paths group-writable because this allows
+# both the dynamo user (UID 1000) and Dev Container users (UID != 1000) to work
+# properly without needing slow chown -R operations (which can add 2-10 extra
+# minutes).
+#
+# DEVELOPMENT PATHS THAT MUST BE GROUP-WRITABLE (for virtualenv containers):
+#   /workspace            - Users create/modify project files
+#   /home/dynamo          - Users create config/cache files
+#   /opt/dynamo/venv      - TensorRT-LLM uses venv, so entire venv must be writable for pip install
+#
+# HOW TO ACHIEVE GROUP-WRITABLE PERMISSIONS:
+# 1. SHELL + /etc/profile.d - Login shell sources umask 002 globally for all RUN commands (775/664)
+# 2. COPY --chmod=775       - Sets permissions on copied children (not destination)
+# 3. chmod g+w (no -R)      - Fixes destination dirs only (milliseconds vs minutes)
 # This section contains build arguments that are common and shared with
 # the plain Dockerfile, so they should NOT have a default. The source of truth is from build.sh.
@@ -238,8 +253,12 @@ RUN userdel -r ubuntu > /dev/null 2>&1 || true \
    && useradd -m -s /bin/bash -g 0 dynamo \
    && [ `id -u dynamo` -eq 1000 ] \
    && mkdir -p /home/dynamo/.cache /opt/dynamo \
-    && chown -R dynamo: /workspace /home/dynamo /opt/dynamo \
+    # Non-recursive chown - only the directories themselves, not contents
-    && chmod -R g+w /workspace /home/dynamo/.cache /opt/dynamo
+    && chown dynamo:0 /home/dynamo /home/dynamo/.cache /opt/dynamo /workspace \
+    # No chmod needed: umask 002 handles new files, COPY --chmod handles copied content
+    # Set umask globally for all subsequent RUN commands (must be done as root before USER dynamo)
+    # NOTE: Setting ENV UMASK=002 does NOT work - umask is a shell builtin, not an environment variable
+    && mkdir -p /etc/profile.d && echo 'umask 002' > /etc/profile.d/00-umask.sh
 # Install Python, build-essential and python3-dev as apt dependencies
 ARG PYTHON_VERSION
@@ -291,6 +310,9 @@ RUN if [ ${ARCH_ALT} = "x86_64" ]; then \
 # Switch to dynamo user
 USER dynamo
 ENV HOME=/home/dynamo
+# This picks up the umask 002 from the /etc/profile.d/00-umask.sh file for subsequent RUN commands
+SHELL ["/bin/bash", "-l", "-o", "pipefail", "-c"]
 ENV DYNAMO_HOME=/workspace
 ENV NIXL_PREFIX=/opt/nvidia/nvda_nixl
 ENV NIXL_LIB_DIR=$NIXL_PREFIX/lib/${ARCH_ALT}-linux-gnu
@@ -301,13 +323,14 @@ COPY --from=framework /usr/local/tensorrt /usr/local/tensorrt
 COPY --from=framework /usr/lib/${ARCH_ALT}-linux-gnu/libgomp.so* /usr/lib/${ARCH_ALT}-linux-gnu/
 # Copy pre-built venv with PyTorch and TensorRT-LLM from framework stage
-COPY --chown=dynamo: --from=framework ${VIRTUAL_ENV} ${VIRTUAL_ENV}
+# Pattern: COPY --chmod=775 <path>; chmod g+w <path> done later as root because COPY --chmod only affects <path>/*, not <path>
+COPY --chmod=775 --chown=dynamo:0 --from=framework ${VIRTUAL_ENV} ${VIRTUAL_ENV}
 # Copy UCX from framework image as plugin for NIXL
 # Copy NIXL source from framework image
-# Copy dynamo wheels for gitlab artifacts
+# Copy dynamo wheels for gitlab artifacts (read-only, no group-write needed)
-COPY --chown=dynamo: --from=dynamo_base /usr/local/ucx /usr/local/ucx
+COPY --chown=dynamo:0 --from=dynamo_base /usr/local/ucx /usr/local/ucx
-COPY --chown=dynamo: --from=dynamo_base $NIXL_PREFIX $NIXL_PREFIX
+COPY --chown=dynamo:0 --from=dynamo_base $NIXL_PREFIX $NIXL_PREFIX
 ENV TENSORRT_LIB_DIR=/usr/local/tensorrt/targets/${ARCH_ALT}-linux-gnu/lib
 ENV PATH="/usr/local/ucx/bin:${VIRTUAL_ENV}/bin:/opt/hpcx/ompi/bin:/usr/local/bin/etcd/:/usr/local/cuda/bin:/usr/local/cuda/nvvm/bin:$PATH"
@@ -324,12 +347,13 @@ $TENSORRT_LIB_DIR:\
 $LD_LIBRARY_PATH
 ENV OPAL_PREFIX=/opt/hpcx/ompi
-COPY --chown=dynamo: ATTRIBUTION* LICENSE /workspace/
+COPY --chmod=664 --chown=dynamo:0 ATTRIBUTION* LICENSE /workspace/
-COPY --chown=dynamo: benchmarks/ /workspace/benchmarks/
+COPY --chmod=775 --chown=dynamo:0 benchmarks/ /workspace/benchmarks/
 # Install dynamo, NIXL, and dynamo-specific dependencies
+# Pattern: COPY --chmod=775 <path>; chmod g+w <path> done later as root because COPY --chmod only affects <path>/*, not <path>
 ARG ENABLE_KVBM
-COPY --chown=dynamo: --from=dynamo_base /opt/dynamo/wheelhouse/ /opt/dynamo/wheelhouse/
+COPY --chmod=775 --chown=dynamo:0 --from=dynamo_base /opt/dynamo/wheelhouse/ /opt/dynamo/wheelhouse/
 RUN uv pip install \
      --no-cache \
      /opt/dynamo/wheelhouse/ai_dynamo_runtime*.whl \
@@ -344,7 +368,9 @@ RUN uv pip install \
        uv pip install --no-cache "$KVBM_WHEEL"; \
    fi && \
    cd /workspace/benchmarks && \
-    UV_GIT_LFS=1 uv pip install --no-cache .
+    UV_GIT_LFS=1 uv pip install --no-cache . && \
+    # pip/uv bypasses umask when creating .egg-info files, but chmod -R is fast here (small directory)
+    chmod -R g+w /workspace/benchmarks
 # Install common and test dependencies
 RUN --mount=type=bind,source=./container/deps/requirements.txt,target=/tmp/requirements.txt \
@@ -357,13 +383,14 @@ RUN --mount=type=bind,source=./container/deps/requirements.txt,target=/tmp/requi
        --requirement /tmp/requirements.test.txt \
        cupy-cuda13x
-# Copy tests, benchmarks, deploy and components for CI with correct ownership
+# Copy tests, deploy and components for CI with correct ownership
-COPY --chown=dynamo: pyproject.toml /workspace/
+# Pattern: COPY --chmod=775 <path>; chmod g+w <path> done later as root because COPY --chmod only affects <path>/*, not <path>
-COPY --chown=dynamo: tests /workspace/tests
+COPY --chmod=775 --chown=dynamo:0 pyproject.toml /workspace/
-COPY --chown=dynamo: examples /workspace/examples
+COPY --chmod=775 --chown=dynamo:0 tests /workspace/tests
-COPY --chown=dynamo: deploy /workspace/deploy
+COPY --chmod=775 --chown=dynamo:0 examples /workspace/examples
-COPY --chown=dynamo: components/ /workspace/components/
+COPY --chmod=775 --chown=dynamo:0 deploy /workspace/deploy
-COPY --chown=dynamo: recipes/ /workspace/recipes/
+COPY --chmod=775 --chown=dynamo:0 components/ /workspace/components/
+COPY --chmod=775 --chown=dynamo:0 recipes/ /workspace/recipes/
 # Setup launch banner in common directory accessible to all users
 RUN --mount=type=bind,source=./container/launch_message/runtime.txt,target=/opt/dynamo/launch_message.txt \
@@ -371,7 +398,10 @@ RUN --mount=type=bind,source=./container/launch_message/runtime.txt,target=/opt/
 # Setup environment for all users
 USER root
-RUN chmod 755 /opt/dynamo/.launch_screen && \
+# Fix directory permissions: COPY --chmod only affects contents, not the directory itself
+RUN chmod g+w ${VIRTUAL_ENV} /workspace /workspace/* /opt/dynamo /opt/dynamo/* && \
+    chown dynamo:0 ${VIRTUAL_ENV} /workspace /opt/dynamo/ && \
+    chmod 755 /opt/dynamo/.launch_screen && \
    echo 'source /opt/dynamo/venv/bin/activate' >> /etc/bash.bashrc && \
    echo 'cat /opt/dynamo/.launch_screen' >> /etc/bash.bashrc
@@ -432,6 +462,10 @@ RUN apt-get update -y && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*
+# Set umask for group-writable files in dev stage (runs as root)
+RUN mkdir -p /etc/profile.d && echo 'umask 002' > /etc/profile.d/00-umask.sh
+SHELL ["/bin/bash", "-l", "-o", "pipefail", "-c"]
 # Set workspace directory variable
 ENV WORKSPACE_DIR=${WORKSPACE_DIR} \
    DYNAMO_HOME=${WORKSPACE_DIR} \
@@ -441,14 +475,17 @@ ENV WORKSPACE_DIR=${WORKSPACE_DIR} \
    VIRTUAL_ENV=/opt/dynamo/venv \
    PATH=/usr/local/cargo/bin:$PATH
-COPY --from=dynamo_base /usr/local/rustup /usr/local/rustup
+# Copy rust installation from dynamo_base to avoid duplication efforts
-COPY --from=dynamo_base /usr/local/cargo /usr/local/cargo
+# Pattern: COPY --chmod=775 <path>; chmod g+w <path> because COPY --chmod only affects <path>/*, not <path>
+COPY --from=dynamo_base --chmod=775 /usr/local/rustup /usr/local/rustup
+COPY --from=dynamo_base --chmod=775 /usr/local/cargo /usr/local/cargo
+RUN chmod g+w /usr/local/rustup /usr/local/cargo
 # Install maturin, for maturin develop
 RUN uv pip install --no-cache maturin[patchelf]
 # Editable install of dynamo
-COPY README.md hatch_build.py /workspace/
+COPY pyproject.toml README.md hatch_build.py /workspace/
 RUN uv pip install --no-cache --no-deps -e .
 CMD []
--- a/container/Dockerfile.vllm
+++ b/container/Dockerfile.vllm
 # syntax=docker/dockerfile:1.10.0
 # SPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 # SPDX-License-Identifier: Apache-2.0
+#
+# Throughout this file, we make certain paths group-writable because this allows
+# both the dynamo user (UID 1000) and Dev Container users (UID != 1000) to work
+# properly without needing slow chown -R operations (which can add 2-10 extra
+# minutes).
+#
+# DEVELOPMENT PATHS THAT MUST BE GROUP-WRITABLE (for virtualenv containers):
+#   /workspace            - Users create/modify project files
+#   /home/dynamo          - Users create config/cache files
+#   /opt/dynamo/venv      - vLLM uses venv, so entire venv must be writable for pip install
+#
+# HOW TO ACHIEVE GROUP-WRITABLE PERMISSIONS:
+# 1. SHELL + /etc/profile.d - Login shell sources umask 002 globally for all RUN commands (775/664)
+# 2. COPY --chmod=775       - Sets permissions on copied children (not destination)
+# 3. chmod g+w (no -R)      - Fixes destination dirs only (milliseconds vs minutes)
 # This section contains build arguments that are common and shared with
 # the plain Dockerfile, so they should NOT have a default. The source of truth is from build.sh.
@@ -210,8 +225,12 @@ RUN userdel -r ubuntu > /dev/null 2>&1 || true \
    && useradd -m -s /bin/bash -g 0 dynamo \
    && [ `id -u dynamo` -eq 1000 ] \
    && mkdir -p /home/dynamo/.cache /opt/dynamo \
-    && chown -R dynamo: /workspace /home/dynamo /opt/dynamo \
+    # Non-recursive chown - only the directories themselves, not contents
-    && chmod -R g+w /workspace /home/dynamo/.cache /opt/dynamo
+    && chown dynamo:0 /home/dynamo /home/dynamo/.cache /opt/dynamo /workspace \
+    # No chmod needed: umask 002 handles new files, COPY --chmod handles copied content
+    # Set umask globally for all subsequent RUN commands (must be done as root before USER dynamo)
+    # NOTE: Setting ENV UMASK=002 does NOT work - umask is a shell builtin, not an environment variable
+    && mkdir -p /etc/profile.d && echo 'umask 002' > /etc/profile.d/00-umask.sh
 ARG ARCH_ALT
 ARG PYTHON_VERSION
@@ -241,20 +260,24 @@ RUN apt-get update && \
 USER dynamo
 ENV HOME=/home/dynamo
+# This picks up the umask 002 from the /etc/profile.d/00-umask.sh file for subsequent RUN commands
+SHELL ["/bin/bash", "-l", "-o", "pipefail", "-c"]
 ENV NIXL_PREFIX=/opt/nvidia/nvda_nixl
 ENV NIXL_LIB_DIR=$NIXL_PREFIX/lib/${ARCH_ALT}-linux-gnu
 ENV NIXL_PLUGIN_DIR=$NIXL_LIB_DIR/plugins
 ### VIRTUAL ENVIRONMENT SETUP ###
 # Copy entire virtual environment from framework container with correct ownership
-COPY --chown=dynamo: --from=framework ${VIRTUAL_ENV} ${VIRTUAL_ENV}
+# Pattern: COPY --chmod=775 <path>; chmod g+w <path> done later as root because COPY --chmod only affects <path>/*, not <path>
+COPY --chmod=775 --chown=dynamo:0 --from=framework ${VIRTUAL_ENV} ${VIRTUAL_ENV}
-# Copy vllm with correct ownership
+# Copy vllm with correct ownership (read-only, no group-write needed)
-COPY --chown=dynamo: --from=framework /opt/vllm /opt/vllm
+COPY --chown=dynamo:0 --from=framework /opt/vllm /opt/vllm
-# Copy UCX and NIXL to system directories
+# Copy UCX and NIXL to system directories (read-only, no group-write needed)
-COPY --chown=dynamo: --from=dynamo_base /usr/local/ucx /usr/local/ucx
+COPY --chown=dynamo:0 --from=dynamo_base /usr/local/ucx /usr/local/ucx
-COPY --chown=dynamo: --from=dynamo_base $NIXL_PREFIX $NIXL_PREFIX
+COPY --chown=dynamo:0 --from=dynamo_base $NIXL_PREFIX $NIXL_PREFIX
 ENV PATH=/usr/local/ucx/bin:$PATH
 ENV LD_LIBRARY_PATH=\
@@ -265,13 +288,15 @@ $NIXL_PLUGIN_DIR:\
 /usr/local/ucx/lib/ucx:\
 $LD_LIBRARY_PATH
-# Copy local files
+# Copy attribution files
-COPY --chown=dynamo: ATTRIBUTION* LICENSE /workspace/
+COPY --chmod=664 --chown=dynamo:0 ATTRIBUTION* LICENSE /workspace/
-COPY --chown=dynamo: benchmarks/ /workspace/benchmarks/
+# Pattern: COPY --chmod=775 <path>; chmod g+w <path> done later as root because COPY --chmod only affects <path>/*, not <path>
+COPY --chmod=775 --chown=dynamo:0 benchmarks/ /workspace/benchmarks/
 # Install dynamo, NIXL, and dynamo-specific dependencies
+# Pattern: COPY --chmod=775 <path>; chmod g+w <path> done later as root because COPY --chmod only affects <path>/*, not <path>
 ARG ENABLE_KVBM
-COPY --chown=dynamo: --from=dynamo_base /opt/dynamo/wheelhouse/ /opt/dynamo/wheelhouse/
+COPY --chmod=775 --chown=dynamo:0 --from=dynamo_base /opt/dynamo/wheelhouse/ /opt/dynamo/wheelhouse/
 RUN uv pip install \
      /opt/dynamo/wheelhouse/ai_dynamo_runtime*.whl \
      /opt/dynamo/wheelhouse/ai_dynamo*any.whl \
@@ -285,7 +310,9 @@ RUN uv pip install \
        uv pip install "$KVBM_WHEEL"; \
    fi && \
    cd /workspace/benchmarks && \
-    UV_GIT_LFS=1 uv pip install --no-cache .
+    UV_GIT_LFS=1 uv pip install --no-cache . && \
+    # pip/uv bypasses umask when creating .egg-info files, but chmod -R is fast here (small directory)
+    chmod -R g+w /workspace/benchmarks
 # Install common and test dependencies
 RUN --mount=type=bind,source=./container/deps/requirements.txt,target=/tmp/requirements.txt \
@@ -295,13 +322,14 @@ RUN --mount=type=bind,source=./container/deps/requirements.txt,target=/tmp/requi
        --requirement /tmp/requirements.txt \
        --requirement /tmp/requirements.test.txt
-# Copy tests, benchmarks, deploy and components for CI
+# Copy tests, deploy and components for CI with correct ownership
-COPY --chown=dynamo: tests /workspace/tests
+# Pattern: COPY --chmod=775 <path>; chmod g+w <path> done later as root because COPY --chmod only affects <path>/*, not <path>
-COPY --chown=dynamo: examples /workspace/examples
+COPY --chmod=775 --chown=dynamo:0 tests /workspace/tests
-COPY --chown=dynamo: deploy /workspace/deploy
+COPY --chmod=775 --chown=dynamo:0 examples /workspace/examples
-COPY --chown=dynamo: recipes/ /workspace/recipes/
+COPY --chmod=775 --chown=dynamo:0 deploy /workspace/deploy
-COPY --chown=dynamo: components/ /workspace/components/
+COPY --chmod=775 --chown=dynamo:0 recipes/ /workspace/recipes/
-COPY --chown=dynamo: lib/ /workspace/lib/
+COPY --chmod=775 --chown=dynamo:0 components/ /workspace/components/
+COPY --chmod=775 --chown=dynamo:0 lib/ /workspace/lib/
 # Setup launch banner in common directory accessible to all users
 RUN --mount=type=bind,source=./container/launch_message/runtime.txt,target=/opt/dynamo/launch_message.txt \
@@ -309,7 +337,9 @@ RUN --mount=type=bind,source=./container/launch_message/runtime.txt,target=/opt/
 # Setup environment for all users
 USER root
-RUN chmod 755 /opt/dynamo/.launch_screen && \
+# Fix directory permissions: COPY --chmod only affects contents, not the directory itself
+RUN chmod g+w /workspace /workspace/* /opt/dynamo /opt/dynamo/* ${VIRTUAL_ENV} && \
+    chmod 755 /opt/dynamo/.launch_screen && \
    echo 'source /opt/dynamo/venv/bin/activate' >> /etc/bash.bashrc && \
    echo 'cat /opt/dynamo/.launch_screen' >> /etc/bash.bashrc
@@ -368,6 +398,10 @@ RUN apt-get update -y && \
    protobuf-compiler && \
    rm -rf /var/lib/apt/lists/*
+# Set umask for group-writable files in dev stage (runs as root)
+RUN mkdir -p /etc/profile.d && echo 'umask 002' > /etc/profile.d/00-umask.sh
+SHELL ["/bin/bash", "-l", "-o", "pipefail", "-c"]
 # Set workspace directory variable
 ENV WORKSPACE_DIR=${WORKSPACE_DIR} \
    DYNAMO_HOME=${WORKSPACE_DIR} \
@@ -377,8 +411,11 @@ ENV WORKSPACE_DIR=${WORKSPACE_DIR} \
    VIRTUAL_ENV=/opt/dynamo/venv \
    PATH=/usr/local/cargo/bin:$PATH
-COPY --from=dynamo_base /usr/local/rustup /usr/local/rustup
+# Copy rust installation from dynamo_base to avoid duplication efforts
-COPY --from=dynamo_base /usr/local/cargo /usr/local/cargo
+# Pattern: COPY --chmod=775 <path>; chmod g+w <path> because COPY --chmod only affects <path>/*, not <path>
+COPY --from=dynamo_base --chmod=775 /usr/local/rustup /usr/local/rustup
+COPY --from=dynamo_base --chmod=775 /usr/local/cargo /usr/local/cargo
+RUN chmod g+w /usr/local/rustup /usr/local/cargo
 # Install maturin, for maturin develop
 # Editable install of dynamo

--- a/container/README.md
+++ b/container/README.md
@@ -37,102 +37,34 @@ The `build.sh` and `run.sh` scripts are convenience wrappers that simplify commo
 ## Development Targets Feature Matrix
-These targets are specified with `build.sh --target <target>` and correspond to Docker multi-stage build targets defined in the Dockerfiles (e.g., `FROM somebase AS <target>`). Some commonly used targets include:
+**Note**: In Dynamo, "targets" and "Docker stages" are synonymous. Each target corresponds to a stage in the multi-stage Docker build. Similarly, "frameworks" and "engines" are synonymous (vLLM, TensorRT-LLM, SGLang).
- `runtime` - For running pre-built containers without development tools (minimal size, runs as non-root `dynamo` user with UID 1000 and GID 0)
+| Feature | **runtime + `run.sh`** | **local-dev (`run.sh` or Dev Container)** | **dev + `run.sh`** (legacy) |
- `dev` - For development (inferencing/benchmarking/etc, runs as root user for maximum flexibility)
+|---------|----------------------|-------------------------------------------|--------------------------|
- `local-dev` - For development with local user permissions matching host UID/GID. This is useful when mounting host partitions (with local user permissions) to Docker partitions. The `dynamo` user UID/GID is remapped to match the host user.
+| **Usage** | Benchmarking inference and deployments, non-root | Development, compilation, testing locally | Legacy workflows, root user, use with caution |
+| **User** | dynamo (UID 1000) | dynamo (UID=host user) with sudo | root (UID 0, use with caution) |
-Additional targets are available in the Dockerfiles for specific build stages and use cases.
+| **Home Directory** | `/home/dynamo` | `/home/dynamo` | `/root` |
+| **Working Directory** | `/workspace` (in-container or mounted) | `/workspace` (must be mounted w/ `--mount-workspace`) | `/workspace` (must be mounted w/ `--mount-workspace`) |
-| Feature | **dev + `run.sh`** | **local-dev + `run.sh`** | **local-dev + Dev Container** |
+| **Rust Toolchain** | None (uses pre-built wheels) | System install (`/usr/local/rustup`, `/usr/local/cargo`) | System install (`/usr/local/rustup`, `/usr/local/cargo`) |
-|---------|-------------------|--------------------------|-------------------------------|
+| **Cargo Target** | None | `/workspace/target` | `/workspace/target` |
-| **Default User** | root | dynamo (matched to host UID/GID) | dynamo (matched to host UID/GID) |
+| **Python Env** | venv (`/opt/dynamo/venv`) for vllm/trtllm, system site-packages for sglang | venv (`/opt/dynamo/venv`) for vllm/trtllm, system site-packages for sglang | venv (`/opt/dynamo/venv`) for vllm/trtllm, system site-packages for sglang |
-| **User Setup** | None (root) | Matches UID/GID of `build.sh` user | Matches UID/GID of `build.sh` user |
-| **Permissions** | root | dynamo with sudo | dynamo with sudo |
-| **Home Directory** | `/root` | `/home/dynamo` | `/home/dynamo` |
-| **Working Directory** | `/workspace` | `/workspace` | `/workspace` |
-| **Rust Toolchain** | System install (`/usr/local/rustup`, `/usr/local/cargo`) | User install (`~/.rustup`, `~/.cargo`) | User install (`~/.rustup`, `~/.cargo`) |
-| **Python Env** | dynamo user owned | dynamo owned venv | dynamo owned venv |
-| **File Permissions** | root-level | user-level (dynamo), safe | user-level (dynamo), safe |
-| **Compatibility** | Legacy workflows, maximum flexibility | workspace writable on NFS, non-root security | workspace writable on NFS, non-root security |
-## Environment Variables Across Build Stages
-Understanding how environment variables change across different build stages is crucial for development and debugging. The Dynamo build system uses a multi-stage Docker build process where environment variables are set, inherited, and overridden at different stages.
-### Build Stage Flow
-```
-Dockerfile → base → dev (dynamo-base image)
-              ↓
-Dockerfile.vllm → framework → runtime → dev (vllm dev image)
-                                         ↓
-Dockerfile.local_dev → local-dev (from vllm dev image)
-```
-### Environment Variables by Stage
-| Variable             | **base**            | **base→dev**         | **vllm→framework** | **vllm→runtime**   | **vllm→dev** | **local-dev** |
-|----------------------|---------------------|----------------------|--------------------|--------------------|--------------|--------------------|
-| **DYNAMO_HOME**      | ❌ Not set          | `/opt/dynamo`        | ❌ Not set         | `/opt/dynamo`      | `/workspace` ✅ **OVERRIDE** | `/workspace` (inherited) |
-| **WORKSPACE_DIR**    | ❌ Not set          | ❌ Not set           | ❌ Not set         | ❌ Not set         | `/workspace` | `/workspace` (inherited) |
-| **CARGO_TARGET_DIR** | ❌ Not set          | `/opt/dynamo/target` | ❌ Not set         | ❌ Not set         | `/workspace/target` ✅ **OVERRIDE** | `/workspace/target` (inherited) |
-| **VIRTUAL_ENV**      | `/opt/dynamo/venv`  | (inherited)          | `/opt/dynamo/venv` | `/opt/dynamo/venv` | `/opt/dynamo/venv` ✅ **REDEFINE** | `/opt/dynamo/venv` (inherited) |
-| **RUSTUP_HOME**      | `/usr/local/rustup` | (inherited)          | ❌ Not set         | ❌ Not set         | `/usr/local/rustup` | `/home/dynamo/.rustup` ✅ **OVERRIDE** |
-| **CARGO_HOME**       | `/usr/local/cargo`  | (inherited)          | ❌ Not set         | ❌ Not set         | `/usr/local/cargo` | `/home/dynamo/.cargo` ✅ **OVERRIDE** |
-| **USERNAME**         | ❌ Not set          | `dynamo`             | ❌ Not set         | `dynamo`           | ❌ Not set   | `dynamo` |
-| **HOME**             | (system default)    | `/home/dynamo`       | (system default)   | `/home/dynamo`     | (system default) | `/home/dynamo` |
-| **PATH**             | (includes cargo)    | (inherited)          | (system default)   | (includes venv, etcd, ucx) | `/usr/local/cargo/bin:$PATH` | `/home/dynamo/.cargo/bin:$PATH` ✅ **OVERRIDE** |
-### Key Insights
-**1. DYNAMO_HOME Dual Purpose:**
- `base→dev` and `vllm→runtime`: `/opt/dynamo` - For **installed/packaged** Dynamo (CI, production)
- `vllm→dev` and `local-dev`: `/workspace` - For **development** with source code mounted from host
-**2. Rust Toolchain Location:**
- `dev` target: System-wide at `/usr/local/rustup` and `/usr/local/cargo` (suitable for root)
- `local-dev` target: User-specific at `/home/dynamo/.rustup` and `/home/dynamo/.cargo` (proper UID/GID ownership)
-**3. Build Artifacts Location:**
- `base→dev`: `/opt/dynamo/target` - Build artifacts with installed package
- `vllm→dev` onward: `/workspace/target` - Build artifacts in mounted workspace for persistence
-**4. Variables That Stay Constant:**
- `VIRTUAL_ENV`: Always `/opt/dynamo/venv` (ownership changes in local-dev via rsync)
- `WORKSPACE_DIR`: Always `/workspace` once set in vllm→dev
- `DYNAMO_HOME`: Always `/workspace` once overridden in vllm→dev (for development)
-**5. local-dev Specific Changes:**
-From `Dockerfile.local_dev`, the Rust toolchain is moved to user home because:
- Workspace mount points may change, breaking toolchain paths
- User needs ownership of cargo binaries and registry for package installation
- Toolchain requires consistent system paths that don't depend on workspace location
-The Python venv ownership is also updated via rsync in local-dev to match the user's UID/GID, ensuring package installation permissions work correctly.
-**6. Non-Root User Architecture:**
-Dynamo containers implement a multi-stage user strategy:
- **runtime stage**: Runs as non-root `dynamo` user (UID 1000, GID 0) for production workloads
- **dev stage**: Runs as root for maximum development flexibility (builds on runtime but switches to root)
- **local-dev stage**: Runs as `dynamo` user with UID/GID matched to host user for safe file system operations
- **Security**: Runtime and local-dev use non-root execution to reduce attack surface
- **File Ownership**: All application files, virtual environments, and build artifacts are owned by `dynamo:root` (1000:0) in runtime stage
- **Environment Setup**: Launch banner moved to `/opt/dynamo/.launch_screen` (shared across all users) and venv activation configured in `/etc/bash.bashrc` for system-wide availability. This replaces the previous per-user `~/.launch_screen` and `~/.bashrc` approach.
 ## Usage Guidelines
- **Use runtime target**: for production deployments. Runs as non-root `dynamo` user (UID 1000, GID 0) for security
+- **Use runtime target**: for benchmarking inference and deployments. Runs as non-root `dynamo` user (UID 1000, GID 0) for security
- **Use dev + `run.sh`**: for command-line testing and inferencing. Runs as root for maximum flexibility
+- **Use local-dev + `run.sh`**: for command-line development and Docker mounted partitions. Runs as `dynamo` user with UID matched to your local user, GID 0. Add `-it` flag for interactive sessions
- **Use local-dev + `run.sh`**: for command-line development and Docker mounted partitions. Runs as `dynamo` user with UID/GID matched to your local user. Add `-it` flag for interactive sessions
+- **Use local-dev + Dev Container**: VS Code/Cursor Dev Container Plugin, using `dynamo` user with UID matched to your local user, GID 0
- **Use local-dev + Dev Container**: VS Code/Cursor Dev Container Plugin, using `dynamo` user with UID/GID matched to your local user
+- **Use dev + `run.sh`**: Root user, use with caution. Runs as root for backward compatibility with early workflows
 ## Example Commands
-### 1. dev + `run.sh` (runs as root):
+### 1. runtime target (runs as non-root dynamo user):
 ```bash
-run.sh ...
+# Build runtime image
+./build.sh --framework vllm --target runtime
+# Run runtime container
+./run.sh --image dynamo:latest-vllm-runtime -it
 ```
 ### 2. local-dev + `run.sh` (runs as dynamo user with matched host UID/GID):
@@ -141,16 +73,7 @@ run.sh --mount-workspace -it --image dynamo:latest-vllm-local-dev ...
 ```
 ### 3. local-dev + Dev Container Extension:
-Use VS Code/Cursor Dev Container Extension with devcontainer.json configuration. The `dynamo` user UID/GID is automatically matched to your local user.
+Use VS Code/Cursor Dev Container Extension with devcontainer.json configuration. The `dynamo` user UID is automatically matched to your local user.
-### 4. runtime target (runs as non-root dynamo user):
-```bash
-# Build runtime image
-./build.sh --framework vllm --target runtime
-# Run runtime container
-./run.sh --image dynamo:latest-vllm-runtime
-```
 ## Build and Run Scripts Overview