Skip to content

GitLab

Unverified Commit 93529753 authored by dagil-nvidia's avatar dagil-nvidia Committed by GitHub
Browse files

fix(examples): add missing --kv-transfer-config to disagg_router.yaml (#6897) 


Signed-off-by: default avatarDan Gil <dagil@nvidia.com>
parent b356b499
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
examples/basics/kubernetes/Distributed_Inference/disagg_router.yaml
View file @ 93529753
...@@ -69,4 +69,4 @@ spec: ...@@ -69,4 +69,4 @@ spec:
- /bin/sh - /bin/sh
- -c - -c
args: args:
- python3 -m dynamo.vllm --model meta-llama/Llama-3.1-70B-Instruct -tp 4 --disaggregation-mode prefill --kv-events-config '{"publisher":"zmq","topic":"kv-events","endpoint":"tcp://*:20080","enable_kv_cache_events":true}' - python3 -m dynamo.vllm --model meta-llama/Llama-3.1-70B-Instruct -tp 4 --disaggregation-mode prefill --kv-transfer-config '{"kv_connector":"NixlConnector","kv_role":"kv_both"}' --kv-events-config '{"publisher":"zmq","topic":"kv-events","endpoint":"tcp://*:20080","enable_kv_cache_events":true}'
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment