Unverified Commit c9d6ff53 authored by Harry Mellor's avatar Harry Mellor Committed by GitHub
Browse files

Explain where the engine args go when using Docker (#12041)


Signed-off-by: default avatarHarry Mellor <19981378+hmellor@users.noreply.github.com>
parent a2d2acb4
...@@ -19,6 +19,8 @@ $ docker run --runtime nvidia --gpus all \ ...@@ -19,6 +19,8 @@ $ docker run --runtime nvidia --gpus all \
--model mistralai/Mistral-7B-v0.1 --model mistralai/Mistral-7B-v0.1
``` ```
You can add any other <project:#engine-args> you need after the image tag (`vllm/vllm-openai:latest`).
```{note} ```{note}
You can either use the `ipc=host` flag or `--shm-size` flag to allow the You can either use the `ipc=host` flag or `--shm-size` flag to allow the
container to access the host's shared memory. vLLM uses PyTorch, which uses shared container to access the host's shared memory. vLLM uses PyTorch, which uses shared
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment