Unverified Commit ac113887 authored by Ying Sheng's avatar Ying Sheng Committed by GitHub
Browse files

Add docker file (#588)


Co-authored-by: default avatarYing Sheng <ying.sheng@databricks.com>
parent dc8cef1d
......@@ -49,6 +49,9 @@ pip install -e "python[all]"
pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.3/
```
### Method 3: Using Docker
The docker images are vailable on Docker Hub as [lmsysorg/sglang](https://hub.docker.com/r/lmsysorg/sglang/tags).
### Notes
- If you see errors from the Triton compiler, please install the [Triton Nightly](https://triton-lang.org/main/getting-started/installation.html).
- If you cannot install FlashInfer, check out its [installation](https://docs.flashinfer.ai/installation.html#) page. If you still cannot install it, you can use the slower Triton kernels by adding `--disable-flashinfer` when launching the server.
......
FROM vllm/vllm-openai
RUN pip install --upgrade pip
RUN pip install "sglang[all]"
RUN pip uninstall -y triton triton-nightly && pip install --no-deps --index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/Triton-Nightly/pypi/simple/ triton-nightly
RUN pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.3/
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment