Skip to content

GitLab

Unverified Commit ebc73f28 authored by Kuntai Du's avatar Kuntai Du Committed by GitHub
Browse files

[Bugfix] Fix a path bug in disaggregated prefill example script. (#12121) 


Signed-off-by: default avatarKuntai Du <kuntai@uchicago.edu>
parent d06e8240
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 1 deletion
examples/online_serving/disaggregated_prefill.sh
View file @ ebc73f28
......@@ -3,6 +3,8 @@
# We will launch 2 vllm instances (1 for prefill and 1 for decode),
# and then transfer the KV cache between them.
set -xe
echo "🚧🚧 Warning: The usage of disaggregated prefill is experimental and subject to change 🚧🚧"
sleep 1
......@@ -69,7 +71,7 @@ wait_for_server 8200
# instance
# NOTE: the usage of this API is subject to change --- in the future we will
# introduce "vllm connect" to connect between prefill and decode instances
python3 ../benchmarks/disagg_benchmarks/disagg_prefill_proxy_server.py &
python3 ../../benchmarks/disagg_benchmarks/disagg_prefill_proxy_server.py &
sleep 1
# serve two example requests
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment