Skip to content

GitLab

Unverified Commit 6306afa6 authored by Ryan McCormick's avatar Ryan McCormick Committed by GitHub
Browse files

fix: Remove asymmetric --request-plane nats from run_engines.sh script (#5245)

parent 535528a5
Show whitespace changes
Inline Side-by-side
Showing with 4 additions and 2 deletions
benchmarks/router/README.md
View file @ 6306afa6
...@@ -107,6 +107,9 @@ We also supports running lightweight mock engines that simulate vLLM behavior wi ...@@ -107,6 +107,9 @@ We also supports running lightweight mock engines that simulate vLLM behavior wi
In a **new terminal**, launch the Dynamo router using the Python CLI: In a **new terminal**, launch the Dynamo router using the Python CLI:
```bash ```bash
# Explicitly set NATS server for KV event publishing
export NATS_SERVER="${NATS_SERVER:-nats://localhost:4222}"
python -m dynamo.frontend \ python -m dynamo.frontend \
--router-mode kv \ --router-mode kv \
--router-reset-states \ --router-reset-states \
......
benchmarks/router/run_engines.sh
View file @ 6306afa6
...@@ -251,7 +251,6 @@ else ...@@ -251,7 +251,6 @@ else
VLLM_ARGS+=("--is-decode-worker") VLLM_ARGS+=("--is-decode-worker")
fi fi
VLLM_ARGS+=("${EXTRA_ARGS[@]}") VLLM_ARGS+=("${EXTRA_ARGS[@]}")
VLLM_ARGS+=("--request-plane" "nats")
exec env PYTHONHASHSEED=0 CUDA_VISIBLE_DEVICES=$GPU_DEVICES DYN_VLLM_KV_EVENT_PORT=$((20080 + i)) VLLM_NIXL_SIDE_CHANNEL_PORT=$((20096 + i)) python3 -m dynamo.vllm \ exec env PYTHONHASHSEED=0 CUDA_VISIBLE_DEVICES=$GPU_DEVICES DYN_VLLM_KV_EVENT_PORT=$((20080 + i)) VLLM_NIXL_SIDE_CHANNEL_PORT=$((20096 + i)) python3 -m dynamo.vllm \
"${VLLM_ARGS[@]}" "${VLLM_ARGS[@]}"
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment