Skip to content

GitLab

Unverified Commit f5e8488c authored by Ziqi Fan's avatar Ziqi Fan Committed by GitHub
Browse files

fix: add VLLM_KV_CAPI_PATH to vllm dockerfile to make kv routing working (#832)

parent 1de737fe
Show whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
container/Dockerfile.vllm
View file @ f5e8488c
...@@ -414,6 +414,8 @@ RUN --mount=type=bind,source=./container/launch_message.txt,target=/workspace/la ...@@ -414,6 +414,8 @@ RUN --mount=type=bind,source=./container/launch_message.txt,target=/workspace/la
sed '/^#\s/d' /workspace/launch_message.txt > ~/.launch_screen && \ sed '/^#\s/d' /workspace/launch_message.txt > ~/.launch_screen && \
echo "cat ~/.launch_screen" >> ~/.bashrc echo "cat ~/.launch_screen" >> ~/.bashrc
# Tell vllm to use the Dynamo LLM C API for KV Cache Routing
ENV VLLM_KV_CAPI_PATH=$CARGO_TARGET_DIR/release/libdynamo_llm_capi.so
########################################## ##########################################
########## Perf Analyzer Image ########### ########## Perf Analyzer Image ###########
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment