Unverified Commit a0a09df0 authored by ishandhanani's avatar ishandhanani Committed by GitHub
Browse files

docs: add an example on how to use `--service-name` flag to spin up a standalone service (#915)

parent 2d39ded6
......@@ -58,6 +58,19 @@ cd $DYNAMO_HOME/examples/llm
dynamo serve components.prefill_worker:PrefillWorker -f ./configs/multinode-405b.yaml
```
**Step 7**: [Optional] Start more decode workers on other nodes
This example can be extended to more nodes as well. For example, if you'd like to spin up another decode worker, you can use
```bash
# node X
export NATS_SERVER = '<your-nats-server-address>' # note this should start with nats://...
export ETCD_ENDPOINTS = '<your-etcd-endpoints-address>'
cd $DYNAMO_HOME/examples/llm
dynamo serve components.worker:VllmWorker -f ./configs/multinode-405b.yaml --service-name VllmWorker
```
Note the use of `--service-name`. This will only spin up the worker that you are requesting and ignore any `depends` statements.
### Client
In another terminal:
......@@ -213,4 +226,4 @@ dynamo serve components.prefill_worker:PrefillWorker -f ./configs/mutinode_disag
In another terminal, you can send the same curl request as described in
[aggregated deployment](#aggregated-deployment), addressing to the ip of
the decode node.
\ No newline at end of file
the decode node.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment