Unverified Commit 9c1d5b45 authored by Nathan Weinberg's avatar Nathan Weinberg Committed by GitHub
Browse files

[Doc] add podman setup instructions for official image (#16796)


Signed-off-by: default avatarNathan Weinberg <nweinber@redhat.com>
parent e31045f9
......@@ -19,6 +19,18 @@ $ docker run --runtime nvidia --gpus all \
--model mistralai/Mistral-7B-v0.1
```
This image can also be used with other container engines such as [Podman](https://podman.io/).
```console
$ podman run --gpus all \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HUGGING_FACE_HUB_TOKEN=$HF_TOKEN" \
-p 8000:8000 \
--ipc=host \
vllm/vllm-openai:latest \
--model mistralai/Mistral-7B-v0.1
```
You can add any other <project:#engine-args> you need after the image tag (`vllm/vllm-openai:latest`).
:::{note}
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment