Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
dynamo
Commits
1c67be28
Unverified
Commit
1c67be28
authored
Apr 11, 2025
by
Tanmay Verma
Committed by
GitHub
Apr 11, 2025
Browse files
build: Define UCX env var to use NVLink when available (#631)
parent
8d35dc43
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
6 additions
and
0 deletions
+6
-0
container/Dockerfile.tensorrt_llm
container/Dockerfile.tensorrt_llm
+6
-0
No files found.
container/Dockerfile.tensorrt_llm
View file @
1c67be28
...
...
@@ -201,6 +201,12 @@ RUN pip install dist/ai_dynamo_runtime*cp312*.whl && \
ENV DYNAMO_KV_CAPI_PATH="/opt/dynamo/bindings/lib/libdynamo_llm_capi.so"
ENV DYNAMO_HOME=/workspace
# Needed to use NVLink for TRTLLM KV Cache Transfer
# https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/advanced/disaggregated-service.md
ENV UCX_CUDA_COPY_ASYNC_MEM_TYPE=cuda
ENV UCX_CUDA_COPY_DMABUF=no
ENV UCX_MEMTYPE_CACHE=no
ENV UCX_RNDV_PIPELINE_ERROR_HANDLING=y
# Copy launch banner
RUN --mount=type=bind,source=./container/launch_message.txt,target=/workspace/launch_message.txt \
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment