Unverified Commit 83a4df04 authored by Hongxia Yang's avatar Hongxia Yang Committed by GitHub
Browse files

[ROCm][Documentation] update quickstart and installation to include rocm...


[ROCm][Documentation] update quickstart and installation to include rocm nightly docker tips (#38367)
Signed-off-by: default avatarHongxia Yang <hongxiay.yang@amd.com>
Co-authored-by: default avatarHongxia Yang <hongxiay.yang@amd.com>
parent 731285c9
...@@ -172,8 +172,11 @@ uv pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/0.15.0/rocm700 ...@@ -172,8 +172,11 @@ uv pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/0.15.0/rocm700
--8<-- [end:build-wheel-from-source] --8<-- [end:build-wheel-from-source]
--8<-- [start:pre-built-images] --8<-- [start:pre-built-images]
vLLM offers an official Docker image for deployment. vLLM offers official Docker images for deployment.
The image can be used to run OpenAI compatible server and is available on Docker Hub as [vllm/vllm-openai-rocm](https://hub.docker.com/r/vllm/vllm-openai-rocm/tags). The images can be used to run OpenAI compatible server and are available on Docker Hub as [vllm/vllm-openai-rocm](https://hub.docker.com/r/vllm/vllm-openai-rocm/tags).
- `vllm/vllm-openai-rocm:latest` — stable release
- `vllm/vllm-openai-rocm:nightly` — preview build from the latest development branch, use this if you want the latest features and fixes
```bash ```bash
docker run --rm \ docker run --rm \
...@@ -186,30 +189,18 @@ docker run --rm \ ...@@ -186,30 +189,18 @@ docker run --rm \
--env "HF_TOKEN=$HF_TOKEN" \ --env "HF_TOKEN=$HF_TOKEN" \
-p 8000:8000 \ -p 8000:8000 \
--ipc=host \ --ipc=host \
vllm/vllm-openai-rocm:latest \ vllm/vllm-openai-rocm:<tag> \
--model Qwen/Qwen3-0.6B --model Qwen/Qwen3-0.6B
``` ```
#### Use AMD's Docker Images #### Use AMD's Docker Images (Deprecated)
Prior to January 20th, 2026 when the official docker images are available on [upstream vLLM docker hub](https://hub.docker.com/v2/repositories/vllm/vllm-openai-rocm/tags/), the [AMD Infinity hub for vLLM](https://hub.docker.com/r/rocm/vllm/tags) offers a prebuilt, optimized !!! warning "Deprecated"
docker image designed for validating inference performance on the AMD Instinct MI300X™ accelerator. AMD's Docker images (`rocm/vllm` and `rocm/vllm-dev`) are deprecated in favor of the official vLLM Docker images above (`vllm/vllm-openai-rocm`). Please migrate to the official images.
AMD also offers nightly prebuilt docker image from [Docker Hub](https://hub.docker.com/r/rocm/vllm-dev), which has vLLM and all its dependencies installed. The entrypoint of this docker image is `/bin/bash` (different from the vLLM's Official Docker Image).
```bash Prior to January 20th, 2026 when the official docker images became available on [upstream vLLM docker hub](https://hub.docker.com/v2/repositories/vllm/vllm-openai-rocm/tags/), the [AMD Infinity hub for vLLM](https://hub.docker.com/r/rocm/vllm/tags) offered a prebuilt, optimized
docker pull rocm/vllm-dev:nightly # to get the latest image docker image designed for validating inference performance on the AMD Instinct MI300X™ accelerator.
docker run -it --rm \ AMD also offered nightly prebuilt docker image from [Docker Hub](https://hub.docker.com/r/rocm/vllm-dev), which has vLLM and all its dependencies installed. The entrypoint of this docker image is `/bin/bash` (different from the vLLM's Official Docker Image).
--network=host \
--group-add=video \
--ipc=host \
--cap-add=SYS_PTRACE \
--security-opt seccomp=unconfined \
--device /dev/kfd \
--device /dev/dri \
-v <path/to/your/models>:/app/models \
-e HF_HOME="/app/models" \
rocm/vllm-dev:nightly
```
!!! tip !!! tip
Please check [LLM inference performance validation on AMD Instinct MI300X](https://rocm.docs.amd.com/en/latest/how-to/performance-validation/mi300x/vllm-benchmark.html) Please check [LLM inference performance validation on AMD Instinct MI300X](https://rocm.docs.amd.com/en/latest/how-to/performance-validation/mi300x/vllm-benchmark.html)
......
...@@ -56,9 +56,12 @@ This guide will help you quickly get started with vLLM to perform: ...@@ -56,9 +56,12 @@ This guide will help you quickly get started with vLLM to perform:
!!! note !!! note
It currently supports Python 3.12, ROCm 7.0 and `glibc >= 2.35`. It currently supports Python 3.12, ROCm 7.0 and `glibc >= 2.35`.
!!! note !!! note
Note that, previously, docker images were published using AMD's docker release pipeline and were located `rocm/vllm-dev`. This is being deprecated by using vLLM's docker release pipeline. Note that, previously, docker images were published using AMD's docker release pipeline and were located `rocm/vllm-dev`. This is being deprecated by using vLLM's docker release pipeline.
!!! tip
A nightly Docker image is also available as [vllm/vllm-openai-rocm:nightly](https://hub.docker.com/r/vllm/vllm-openai-rocm/tags) for testing the latest development builds.
=== "Google TPU" === "Google TPU"
To run vLLM on Google TPUs, you need to install the `vllm-tpu` package. To run vLLM on Google TPUs, you need to install the `vllm-tpu` package.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment