Skip to content

GitLab

Unverified Commit 79ad7f36 authored by ishandhanani's avatar ishandhanani Committed by GitHub
Browse files

fix: disagg dp attention needs host 0.0.0.0 (#3809)

parent db8e52f2
Show whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
components/backends/sglang/launch/disagg_dp_attn.sh
View file @ 79ad7f36
...@@ -24,6 +24,7 @@ python3 -m dynamo.sglang \ ...@@ -24,6 +24,7 @@ python3 -m dynamo.sglang \
--dp-size 2 \ --dp-size 2 \
--page-size 16 \ --page-size 16 \
--enable-dp-attention \ --enable-dp-attention \
--host 0.0.0.0 \
--trust-remote-code \ --trust-remote-code \
--disaggregation-mode prefill \ --disaggregation-mode prefill \
--disaggregation-transfer-backend nixl \ --disaggregation-transfer-backend nixl \
...@@ -39,6 +40,7 @@ CUDA_VISIBLE_DEVICES=2,3 python3 -m dynamo.sglang \ ...@@ -39,6 +40,7 @@ CUDA_VISIBLE_DEVICES=2,3 python3 -m dynamo.sglang \
--dp-size 2 \ --dp-size 2 \
--page-size 16 \ --page-size 16 \
--enable-dp-attention \ --enable-dp-attention \
--host 0.0.0.0 \
--trust-remote-code \ --trust-remote-code \
--disaggregation-mode decode \ --disaggregation-mode decode \
--disaggregation-transfer-backend nixl \ --disaggregation-transfer-backend nixl \
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment