Unverified Commit 2c3656f2 authored by shizhediao's avatar shizhediao Committed by GitHub
Browse files

[Fix Doc.] Enable internal forwarding when starting the router (#4355)

parent d40ee62b
...@@ -27,7 +27,7 @@ The router supports two working modes: ...@@ -27,7 +27,7 @@ The router supports two working modes:
This will be a drop-in replacement for the existing `--dp-size` argument of SGLang Runtime. Under the hood, it uses multi-processes to launch multiple workers, wait for them to be ready, then connect the router to all workers. This will be a drop-in replacement for the existing `--dp-size` argument of SGLang Runtime. Under the hood, it uses multi-processes to launch multiple workers, wait for them to be ready, then connect the router to all workers.
```bash ```bash
python -m sglang_router.launch_server --model-path meta-llama/Meta-Llama-3.1-8B-Instruct --dp-size 4 python -m sglang_router.launch_server --model-path meta-llama/Meta-Llama-3.1-8B-Instruct --dp-size 4 --host 0.0.0.0
``` ```
After the server is ready, you can directly send requests to the router as the same way as sending requests to each single worker. After the server is ready, you can directly send requests to the router as the same way as sending requests to each single worker.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment