Skip to content

GitLab

Unverified Commit 262cce76 authored by Kris Hung's avatar Kris Hung Committed by GitHub
Browse files

fix: Update agg_kvbm_router.sh to run workers on two gpus (#4643)

parent 01a634d1
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
examples/backends/vllm/launch/agg_kvbm_router.sh
View file @ 262cce76
...@@ -32,7 +32,7 @@ CUDA_VISIBLE_DEVICES=0 DYN_KVBM_CPU_CACHE_GB=2 \ ...@@ -32,7 +32,7 @@ CUDA_VISIBLE_DEVICES=0 DYN_KVBM_CPU_CACHE_GB=2 \
DYN_KVBM_LEADER_ZMQ_PUB_PORT=56003 \ DYN_KVBM_LEADER_ZMQ_PUB_PORT=56003 \
DYN_KVBM_LEADER_ZMQ_ACK_PORT=56004 \ DYN_KVBM_LEADER_ZMQ_ACK_PORT=56004 \
VLLM_NIXL_SIDE_CHANNEL_PORT=20097 \ VLLM_NIXL_SIDE_CHANNEL_PORT=20097 \
CUDA_VISIBLE_DEVICES=0 DYN_KVBM_CPU_CACHE_GB=2 \ CUDA_VISIBLE_DEVICES=1 DYN_KVBM_CPU_CACHE_GB=2 \
python3 -m dynamo.vllm \ python3 -m dynamo.vllm \
--model $MODEL \ --model $MODEL \
--enforce-eager \ --enforce-eager \
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment