Unverified Commit 4ad281f2 authored by Tanmay Verma's avatar Tanmay Verma Committed by GitHub
Browse files

refactor: Move TRTLLM example to the component/backends (#1976)

parent 57d24a18
......@@ -43,7 +43,7 @@ inside an interactive shell on one of the allocated nodes, set the
following environment variables based:
```bash
cd $DYNAMO_ROOT/examples/tensorrt_llm
cd $DYNAMO_HOME/components/backends/trtllm
export IMAGE="<dynamo_trtllm_image>"
# export MOUNTS="${PWD}/:/mnt,/lustre:/lustre"
......
......@@ -48,7 +48,7 @@ For simplicity of the example, we will make some assumptions about your slurm cl
If your cluster supports similar container based plugins, you may be able to
modify the script to use that instead.
3. Third, we assume you have already built a recent Dynamo+TRTLLM container image as
described [here](https://github.com/ai-dynamo/dynamo/tree/main/examples/tensorrt_llm#build-docker).
described [here](https://github.com/ai-dynamo/dynamo/tree/main/components/backends/trtllm#build-docker).
This is the image that can be set to the `IMAGE` environment variable in later steps.
4. Fourth, we assume you pre-allocate a group of nodes using `salloc`. We
will allocate 8 nodes below as a reference command to have enough capacity
......@@ -87,7 +87,7 @@ following environment variables based:
```bash
# NOTE: IMAGE must be set manually for now
# To build an iamge, see the steps here:
# https://github.com/ai-dynamo/dynamo/tree/main/examples/tensorrt_llm#build-docker
# https://github.com/ai-dynamo/dynamo/tree/main/components/backends/trtllm#build-docker
export IMAGE="<dynamo_trtllm_image>"
# MOUNTS are the host:container path pairs that are mounted into the containers
......
......@@ -32,7 +32,7 @@ if [[ -z ${IMAGE} ]]; then
echo "ERROR: You need to set the IMAGE environment variable to the " \
"Dynamo+TRTLLM docker image or .sqsh file from 'enroot import' " \
"See how to build one from source here: " \
"https://github.com/ai-dynamo/dynamo/tree/main/examples/tensorrt_llm#build-docker"
"https://github.com/ai-dynamo/dynamo/tree/main/components/backends/trtllm#build-docker"
exit 1
fi
......
......@@ -35,7 +35,7 @@ if [[ -z ${IMAGE} ]]; then
echo "ERROR: You need to set the IMAGE environment variable to the " \
"Dynamo+TRTLLM docker image or .sqsh file from 'enroot import' " \
"See how to build one from source here: " \
"https://github.com/ai-dynamo/dynamo/tree/main/examples/tensorrt_llm#build-docker"
"https://github.com/ai-dynamo/dynamo/tree/main/components/backends/trtllm#build-docker"
exit 1
fi
......
......@@ -484,11 +484,11 @@ ENV TRTLLM_USE_UCX_KVCACHE=1
COPY --from=dev /usr/local/bin/set_trtllm_env.sh /usr/local/bin/set_trtllm_env.sh
RUN echo 'source /usr/local/bin/set_trtllm_env.sh' >> /root/.bashrc
# Copy benchmarks, examples and tests for CI
# Copy benchmarks, backends and tests for CI
# TODO: Remove this once we have a functional CI image built on top of the runtime image
COPY tests /workspace/tests
COPY benchmarks /workspace/benchmarks
COPY examples /workspace/examples
COPY components/backends/trtllm /workspace/components/backends/trtllm
RUN uv pip install /workspace/benchmarks
# Copy files for legal compliance
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment