Unverified Commit 01426695 authored by Ziqi Fan's avatar Ziqi Fan Committed by GitHub
Browse files

docs: update kvbm guide according to docker build change (#6264)


Signed-off-by: default avatarZiqi Fan <ziqif@nvidia.com>
parent 40355b32
...@@ -49,10 +49,11 @@ To build KVBM from source, see the detailed instructions in the [KVBM bindings R ...@@ -49,10 +49,11 @@ To build KVBM from source, see the detailed instructions in the [KVBM bindings R
docker compose -f deploy/docker-compose.yml up -d docker compose -f deploy/docker-compose.yml up -d
# Build a dynamo vLLM container (KVBM is built in by default) # Build a dynamo vLLM container (KVBM is built in by default)
./container/build.sh --framework vllm python container/render.py --framework vllm --target runtime --output-short-filename
docker build -t dynamo:latest-vllm-runtime -f container/rendered.Dockerfile .
# Launch the container # Launch the container
./container/run.sh --framework vllm -it --mount-workspace --use-nixl-gds container/run.sh --image dynamo:latest-vllm-runtime -it --mount-workspace --use-nixl-gds
``` ```
### Aggregated Serving ### Aggregated Serving
...@@ -99,10 +100,11 @@ vllm serve --kv-transfer-config '{"kv_connector":"DynamoConnector","kv_role":"kv ...@@ -99,10 +100,11 @@ vllm serve --kv-transfer-config '{"kv_connector":"DynamoConnector","kv_role":"kv
docker compose -f deploy/docker-compose.yml up -d docker compose -f deploy/docker-compose.yml up -d
# Build a dynamo TRTLLM container (KVBM is built in by default) # Build a dynamo TRTLLM container (KVBM is built in by default)
./container/build.sh --framework trtllm python container/render.py --framework trtllm --target runtime --output-short-filename
docker build -t dynamo:latest-trtllm-runtime -f container/rendered.Dockerfile .
# Launch the container # Launch the container
./container/run.sh --framework trtllm -it --mount-workspace --use-nixl-gds container/run.sh --image dynamo:latest-trtllm-runtime -it --mount-workspace --use-nixl-gds
``` ```
### Aggregated Serving ### Aggregated Serving
......
...@@ -43,7 +43,7 @@ The pip wheel is built through a Docker build process: ...@@ -43,7 +43,7 @@ The pip wheel is built through a Docker build process:
```bash ```bash
# Render and build the Docker image with KVBM enabled (from the dynamo repo root) # Render and build the Docker image with KVBM enabled (from the dynamo repo root)
python container/render.py --framework=dynamo --target==runtime --output-short-filename python container/render.py --framework dynamo --target runtime --output-short-filename
docker build --build-arg ENABLE_KVBM="true" -f container/rendered.Dockerfile . docker build --build-arg ENABLE_KVBM="true" -f container/rendered.Dockerfile .
``` ```
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment