Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
fb58e3a6
Unverified
Commit
fb58e3a6
authored
Jul 29, 2025
by
Michael Goin
Committed by
GitHub
Jul 29, 2025
Browse files
[Docs] Update docker.md with HF_TOKEN, new model, and podman fix (#21856)
parent
76080cff
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
5 deletions
+5
-5
docs/deployment/docker.md
docs/deployment/docker.md
+5
-5
No files found.
docs/deployment/docker.md
View file @
fb58e3a6
...
...
@@ -10,23 +10,23 @@ The image can be used to run OpenAI compatible server and is available on Docker
```
bash
docker run
--runtime
nvidia
--gpus
all
\
-v
~/.cache/huggingface:/root/.cache/huggingface
\
--env
"HUGGING_FACE_HUB_TOKEN=
<secret>
"
\
--env
"HUGGING_FACE_HUB_TOKEN=
$HF_TOKEN
"
\
-p
8000:8000
\
--ipc
=
host
\
vllm/vllm-openai:latest
\
--model
mistralai/Mistral-7B-v0.1
--model
Qwen/Qwen3-0.6B
```
This image can also be used with other container engines such as
[
Podman
](
https://podman.io/
)
.
```
bash
podman run
--gpu
s
all
\
podman run
--
device
nvidia.com/
gpu
=
all
\
-v
~/.cache/huggingface:/root/.cache/huggingface
\
--env
"HUGGING_FACE_HUB_TOKEN=
$HF_TOKEN
"
\
-p
8000:8000
\
--ipc
=
host
\
vllm/vllm-openai:latest
\
--model
mistralai/Mistral-7B-v0.1
docker.io/
vllm/vllm-openai:latest
\
--model
Qwen/Qwen3-0.6B
```
You can add any other
[
engine-args
](
../configuration/engine_args.md
)
you need after the image tag (
`vllm/vllm-openai:latest`
).
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment