Unverified Commit 4ad281f2 authored by Tanmay Verma's avatar Tanmay Verma Committed by GitHub
Browse files

refactor: Move TRTLLM example to the component/backends (#1976)

parent 57d24a18
...@@ -43,7 +43,7 @@ inside an interactive shell on one of the allocated nodes, set the ...@@ -43,7 +43,7 @@ inside an interactive shell on one of the allocated nodes, set the
following environment variables based: following environment variables based:
```bash ```bash
cd $DYNAMO_ROOT/examples/tensorrt_llm cd $DYNAMO_HOME/components/backends/trtllm
export IMAGE="<dynamo_trtllm_image>" export IMAGE="<dynamo_trtllm_image>"
# export MOUNTS="${PWD}/:/mnt,/lustre:/lustre" # export MOUNTS="${PWD}/:/mnt,/lustre:/lustre"
......
...@@ -48,7 +48,7 @@ For simplicity of the example, we will make some assumptions about your slurm cl ...@@ -48,7 +48,7 @@ For simplicity of the example, we will make some assumptions about your slurm cl
If your cluster supports similar container based plugins, you may be able to If your cluster supports similar container based plugins, you may be able to
modify the script to use that instead. modify the script to use that instead.
3. Third, we assume you have already built a recent Dynamo+TRTLLM container image as 3. Third, we assume you have already built a recent Dynamo+TRTLLM container image as
described [here](https://github.com/ai-dynamo/dynamo/tree/main/examples/tensorrt_llm#build-docker). described [here](https://github.com/ai-dynamo/dynamo/tree/main/components/backends/trtllm#build-docker).
This is the image that can be set to the `IMAGE` environment variable in later steps. This is the image that can be set to the `IMAGE` environment variable in later steps.
4. Fourth, we assume you pre-allocate a group of nodes using `salloc`. We 4. Fourth, we assume you pre-allocate a group of nodes using `salloc`. We
will allocate 8 nodes below as a reference command to have enough capacity will allocate 8 nodes below as a reference command to have enough capacity
...@@ -87,7 +87,7 @@ following environment variables based: ...@@ -87,7 +87,7 @@ following environment variables based:
```bash ```bash
# NOTE: IMAGE must be set manually for now # NOTE: IMAGE must be set manually for now
# To build an iamge, see the steps here: # To build an iamge, see the steps here:
# https://github.com/ai-dynamo/dynamo/tree/main/examples/tensorrt_llm#build-docker # https://github.com/ai-dynamo/dynamo/tree/main/components/backends/trtllm#build-docker
export IMAGE="<dynamo_trtllm_image>" export IMAGE="<dynamo_trtllm_image>"
# MOUNTS are the host:container path pairs that are mounted into the containers # MOUNTS are the host:container path pairs that are mounted into the containers
......
...@@ -32,7 +32,7 @@ if [[ -z ${IMAGE} ]]; then ...@@ -32,7 +32,7 @@ if [[ -z ${IMAGE} ]]; then
echo "ERROR: You need to set the IMAGE environment variable to the " \ echo "ERROR: You need to set the IMAGE environment variable to the " \
"Dynamo+TRTLLM docker image or .sqsh file from 'enroot import' " \ "Dynamo+TRTLLM docker image or .sqsh file from 'enroot import' " \
"See how to build one from source here: " \ "See how to build one from source here: " \
"https://github.com/ai-dynamo/dynamo/tree/main/examples/tensorrt_llm#build-docker" "https://github.com/ai-dynamo/dynamo/tree/main/components/backends/trtllm#build-docker"
exit 1 exit 1
fi fi
......
...@@ -35,7 +35,7 @@ if [[ -z ${IMAGE} ]]; then ...@@ -35,7 +35,7 @@ if [[ -z ${IMAGE} ]]; then
echo "ERROR: You need to set the IMAGE environment variable to the " \ echo "ERROR: You need to set the IMAGE environment variable to the " \
"Dynamo+TRTLLM docker image or .sqsh file from 'enroot import' " \ "Dynamo+TRTLLM docker image or .sqsh file from 'enroot import' " \
"See how to build one from source here: " \ "See how to build one from source here: " \
"https://github.com/ai-dynamo/dynamo/tree/main/examples/tensorrt_llm#build-docker" "https://github.com/ai-dynamo/dynamo/tree/main/components/backends/trtllm#build-docker"
exit 1 exit 1
fi fi
......
...@@ -484,11 +484,11 @@ ENV TRTLLM_USE_UCX_KVCACHE=1 ...@@ -484,11 +484,11 @@ ENV TRTLLM_USE_UCX_KVCACHE=1
COPY --from=dev /usr/local/bin/set_trtllm_env.sh /usr/local/bin/set_trtllm_env.sh COPY --from=dev /usr/local/bin/set_trtllm_env.sh /usr/local/bin/set_trtllm_env.sh
RUN echo 'source /usr/local/bin/set_trtllm_env.sh' >> /root/.bashrc RUN echo 'source /usr/local/bin/set_trtllm_env.sh' >> /root/.bashrc
# Copy benchmarks, examples and tests for CI # Copy benchmarks, backends and tests for CI
# TODO: Remove this once we have a functional CI image built on top of the runtime image # TODO: Remove this once we have a functional CI image built on top of the runtime image
COPY tests /workspace/tests COPY tests /workspace/tests
COPY benchmarks /workspace/benchmarks COPY benchmarks /workspace/benchmarks
COPY examples /workspace/examples COPY components/backends/trtllm /workspace/components/backends/trtllm
RUN uv pip install /workspace/benchmarks RUN uv pip install /workspace/benchmarks
# Copy files for legal compliance # Copy files for legal compliance
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment