Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
9c1d5b45
Unverified
Commit
9c1d5b45
authored
Apr 18, 2025
by
Nathan Weinberg
Committed by
GitHub
Apr 18, 2025
Browse files
[Doc] add podman setup instructions for official image (#16796)
Signed-off-by:
Nathan Weinberg
<
nweinber@redhat.com
>
parent
e31045f9
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
12 additions
and
0 deletions
+12
-0
docs/source/deployment/docker.md
docs/source/deployment/docker.md
+12
-0
No files found.
docs/source/deployment/docker.md
View file @
9c1d5b45
...
...
@@ -19,6 +19,18 @@ $ docker run --runtime nvidia --gpus all \
--model
mistralai/Mistral-7B-v0.1
```
This image can also be used with other container engines such as
[
Podman
](
https://podman.io/
)
.
```
console
$
podman run
--gpus
all
\
-v
~/.cache/huggingface:/root/.cache/huggingface
\
--env
"HUGGING_FACE_HUB_TOKEN=
$HF_TOKEN
"
\
-p
8000:8000
\
--ipc
=
host
\
vllm/vllm-openai:latest
\
--model
mistralai/Mistral-7B-v0.1
```
You can add any other
<project:
#engine-args
>
you need after the image tag (
`vllm/vllm-openai:latest`
).
:::{note}
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment