Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
dynamo
Commits
b2044566
"vscode:/vscode.git/clone" did not exist on "b5fddbd0095f88f045ba1b07dd652d02f3130f8f"
Unverified
Commit
b2044566
authored
Jul 07, 2025
by
Tanmay Verma
Committed by
GitHub
Jul 07, 2025
Browse files
fix: Set TRTLLM_USE_UCX_KVCACHE by default in container image (#1777)
parent
1630f8ba
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
6 additions
and
2 deletions
+6
-2
container/Dockerfile.tensorrt_llm
container/Dockerfile.tensorrt_llm
+6
-2
No files found.
container/Dockerfile.tensorrt_llm
View file @
b2044566
...
...
@@ -329,13 +329,17 @@ ENV LD_LIBRARY_PATH=/usr/local/nixl/lib/${ARCH_ALT}-linux-gnu:$LD_LIBRARY_PATH
# Use UCX for TRTLLM KV Cache Transfer
ARG TRTLLM_USE_NIXL_KVCACHE_EXPERIMENTAL
ENV TRTLLM_USE_UCX_KVCACHE=1
# Create a script that sets the environment variables and source it
# Create a script that sets the environment variables and source it.
# If TRTLLM_USE_NIXL_KVCACHE_EXPERIMENTAL is set to 1, unset TRTLLM_USE_UCX_KVCACHE
# and set TRTLLM_USE_NIXL_KVCACHE to 1.
RUN echo '#!/bin/bash' > /usr/local/bin/set_trtllm_env.sh && \
if [ "$TRTLLM_USE_NIXL_KVCACHE_EXPERIMENTAL" = "1" ]; then \
echo 'unset TRTLLM_USE_UCX_KVCACHE' >> /usr/local/bin/set_trtllm_env.sh; \
echo 'export TRTLLM_USE_NIXL_KVCACHE=1' >> /usr/local/bin/set_trtllm_env.sh; \
else \
echo '
export
TRTLLM_USE_UCX_KVCACHE
=
1' >> /usr/local/bin/set_trtllm_env.sh; \
echo '
#
TRTLLM_USE_UCX_KVCACHE
should already be set to
1' >> /usr/local/bin/set_trtllm_env.sh; \
fi && \
chmod +x /usr/local/bin/set_trtllm_env.sh
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment