Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
ebc73f28
Unverified
Commit
ebc73f28
authored
Jan 17, 2025
by
Kuntai Du
Committed by
GitHub
Jan 17, 2025
Browse files
[Bugfix] Fix a path bug in disaggregated prefill example script. (#12121)
Signed-off-by:
Kuntai Du
<
kuntai@uchicago.edu
>
parent
d06e8240
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
1 deletion
+3
-1
examples/online_serving/disaggregated_prefill.sh
examples/online_serving/disaggregated_prefill.sh
+3
-1
No files found.
examples/online_serving/disaggregated_prefill.sh
View file @
ebc73f28
...
...
@@ -3,6 +3,8 @@
# We will launch 2 vllm instances (1 for prefill and 1 for decode),
# and then transfer the KV cache between them.
set
-xe
echo
"🚧🚧 Warning: The usage of disaggregated prefill is experimental and subject to change 🚧🚧"
sleep
1
...
...
@@ -69,7 +71,7 @@ wait_for_server 8200
# instance
# NOTE: the usage of this API is subject to change --- in the future we will
# introduce "vllm connect" to connect between prefill and decode instances
python3 ../benchmarks/disagg_benchmarks/disagg_prefill_proxy_server.py &
python3
../
../benchmarks/disagg_benchmarks/disagg_prefill_proxy_server.py &
sleep
1
# serve two example requests
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment