Skip to content

GitLab

Unverified Commit 6c870810 authored by hhzhang16's avatar hhzhang16 Committed by GitHub
Browse files

fix: add --is-decode-worker flag (#5843) 


Signed-off-by: default avatarHannah Zhang <hannahz@nvidia.com>
parent b88cb59b
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
examples/basics/kubernetes/Distributed_Inference/disagg_router.yaml
View file @ 6c870810
......@@ -42,7 +42,7 @@ spec:
- /bin/sh
- -c
args:
- python3 -m dynamo.vllm --model meta-llama/Llama-3.1-70B-Instruct -tp 8
- python3 -m dynamo.vllm --model meta-llama/Llama-3.1-70B-Instruct -tp 8 --is-decode-worker
VllmPrefillWorker:
envFromSecret: hf-token-secret
componentType: worker
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment