Unverified Commit fb58e3a6 authored by Michael Goin's avatar Michael Goin Committed by GitHub
Browse files

[Docs] Update docker.md with HF_TOKEN, new model, and podman fix (#21856)

parent 76080cff
...@@ -10,23 +10,23 @@ The image can be used to run OpenAI compatible server and is available on Docker ...@@ -10,23 +10,23 @@ The image can be used to run OpenAI compatible server and is available on Docker
```bash ```bash
docker run --runtime nvidia --gpus all \ docker run --runtime nvidia --gpus all \
-v ~/.cache/huggingface:/root/.cache/huggingface \ -v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HUGGING_FACE_HUB_TOKEN=<secret>" \ --env "HUGGING_FACE_HUB_TOKEN=$HF_TOKEN" \
-p 8000:8000 \ -p 8000:8000 \
--ipc=host \ --ipc=host \
vllm/vllm-openai:latest \ vllm/vllm-openai:latest \
--model mistralai/Mistral-7B-v0.1 --model Qwen/Qwen3-0.6B
``` ```
This image can also be used with other container engines such as [Podman](https://podman.io/). This image can also be used with other container engines such as [Podman](https://podman.io/).
```bash ```bash
podman run --gpus all \ podman run --device nvidia.com/gpu=all \
-v ~/.cache/huggingface:/root/.cache/huggingface \ -v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HUGGING_FACE_HUB_TOKEN=$HF_TOKEN" \ --env "HUGGING_FACE_HUB_TOKEN=$HF_TOKEN" \
-p 8000:8000 \ -p 8000:8000 \
--ipc=host \ --ipc=host \
vllm/vllm-openai:latest \ docker.io/vllm/vllm-openai:latest \
--model mistralai/Mistral-7B-v0.1 --model Qwen/Qwen3-0.6B
``` ```
You can add any other [engine-args](../configuration/engine_args.md) you need after the image tag (`vllm/vllm-openai:latest`). You can add any other [engine-args](../configuration/engine_args.md) you need after the image tag (`vllm/vllm-openai:latest`).
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment