Unverified Commit f17aec0d authored by Reid's avatar Reid Committed by GitHub
Browse files

[doc] Fold long code blocks to improve readability (#19926)


Signed-off-by: default avatarreidliu41 <reid201711@gmail.com>
Co-authored-by: default avatarreidliu41 <reid201711@gmail.com>
parent 493c2753
...@@ -90,24 +90,26 @@ Currently, there are no pre-built ROCm wheels. ...@@ -90,24 +90,26 @@ Currently, there are no pre-built ROCm wheels.
4. Build vLLM. For example, vLLM on ROCM 6.3 can be built with the following steps: 4. Build vLLM. For example, vLLM on ROCM 6.3 can be built with the following steps:
```bash ??? Commands
pip install --upgrade pip
```bash
# Build & install AMD SMI pip install --upgrade pip
pip install /opt/rocm/share/amd_smi
# Build & install AMD SMI
# Install dependencies pip install /opt/rocm/share/amd_smi
pip install --upgrade numba \
scipy \ # Install dependencies
huggingface-hub[cli,hf_transfer] \ pip install --upgrade numba \
setuptools_scm scipy \
pip install "numpy<2" huggingface-hub[cli,hf_transfer] \
pip install -r requirements/rocm.txt setuptools_scm
pip install "numpy<2"
# Build vLLM for MI210/MI250/MI300. pip install -r requirements/rocm.txt
export PYTORCH_ROCM_ARCH="gfx90a;gfx942"
python3 setup.py develop # Build vLLM for MI210/MI250/MI300.
``` export PYTORCH_ROCM_ARCH="gfx90a;gfx942"
python3 setup.py develop
```
This may take 5-10 minutes. Currently, `pip install .` does not work for ROCm installation. This may take 5-10 minutes. Currently, `pip install .` does not work for ROCm installation.
...@@ -201,19 +203,21 @@ DOCKER_BUILDKIT=1 docker build \ ...@@ -201,19 +203,21 @@ DOCKER_BUILDKIT=1 docker build \
To run the above docker image `vllm-rocm`, use the below command: To run the above docker image `vllm-rocm`, use the below command:
```console ??? Command
docker run -it \
--network=host \ ```console
--group-add=video \ docker run -it \
--ipc=host \ --network=host \
--cap-add=SYS_PTRACE \ --group-add=video \
--security-opt seccomp=unconfined \ --ipc=host \
--device /dev/kfd \ --cap-add=SYS_PTRACE \
--device /dev/dri \ --security-opt seccomp=unconfined \
-v <path/to/model>:/app/model \ --device /dev/kfd \
vllm-rocm \ --device /dev/dri \
bash -v <path/to/model>:/app/model \
``` vllm-rocm \
bash
```
Where the `<path/to/model>` is the location where the model is stored, for example, the weights for llama2 or llama3 models. Where the `<path/to/model>` is the location where the model is stored, for example, the weights for llama2 or llama3 models.
......
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment