"examples/llm/git@developer.sourcefind.cn:OpenDAS/dynamo.git" did not exist on "4b1867c53ebbf98dea54623af24d2424ead56573"
Commit df54b9cb authored by tlipoca9's avatar tlipoca9 Committed by GitHub
Browse files

docs: update close-deployment in dynamo_serve.md (#535)


Co-authored-by: default avatarishandhanani <82981111+ishandhanani@users.noreply.github.com>
parent c1c875bb
...@@ -235,20 +235,10 @@ curl localhost:8000/v1/chat/completions -H "Content-Type: application/json" ...@@ -235,20 +235,10 @@ curl localhost:8000/v1/chat/completions -H "Content-Type: application/json"
}' }'
``` ```
## Close your deployment ## Close deployment
If you have any lingering processes after running `ctrl-c`, you can kill them by running > [!IMPORTANT]
> We are aware of an issue where vLLM subprocesses might not be killed when `ctrl-c` is pressed.
> We are working on addressing this. Relevant vLLM issues can be found [here](https://github.com/vllm-project/vllm/pull/8492) and [here](https://github.com/vllm-project/vllm/issues/6219#issuecomment-2439257824).
```bash To stop the serve, you can press `ctrl-c` which will kill the different components. In order to kill the remaining vLLM subprocesses you can run `nvidia-smi` and `kill -9` the remaining processes or run `pkill python3` from inside of the container.
function kill_tree() {
local parent=$1
local children=$(ps -o pid= --ppid $parent)
for child in $children; do
kill_tree $child
done
echo "Killing process $parent"
kill -9 $parent
}
kill_tree $(pgrep circusd)
```
...@@ -157,8 +157,4 @@ See [multinode-examples.md](multinode-examples.md) for more details. ...@@ -157,8 +157,4 @@ See [multinode-examples.md](multinode-examples.md) for more details.
### Close deployment ### Close deployment
> [!IMPORTANT] See [close deployment](../../docs/guides/dynamo_serve.md#close-deployment) section to learn about how to close the deployment.
> We are aware of an issue where vLLM subprocesses might not be killed when `ctrl-c` is pressed.
> We are working on addressing this. Relevant vLLM issues can be found [here](https://github.com/vllm-project/vllm/pull/8492) and [here](https://github.com/vllm-project/vllm/issues/6219#issuecomment-2439257824).
To stop the serve, you can press `ctrl-c` which will kill the different components. In order to kill the remaining vLLM subprocesses you can run `nvidia-smi` and `kill -9` the remaining processes or run `pkill python3` from inside of the container.
\ No newline at end of file
...@@ -119,7 +119,7 @@ See [client](../llm/README.md#client) section to learn how to send request to th ...@@ -119,7 +119,7 @@ See [client](../llm/README.md#client) section to learn how to send request to th
### Close deployment ### Close deployment
See [close deployment](../llm/README.md#close-deployment) section to learn about how to close the deployment. See [close deployment](../../docs/guides/dynamo_serve.md#close-deployment) section to learn about how to close the deployment.
Remaining tasks: Remaining tasks:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment