Skip to content

GitLab

Unverified Commit 6186a8f8 authored by Yineng Zhang's avatar Yineng Zhang Committed by GitHub
Browse files

update flashinfer install index url (#3293)

parent a07364cc
Show whitespace changes
Inline Side-by-side
Showing with 10 additions and 10 deletions
docker/Dockerfile
View file @ 6186a8f8
...@@ -41,26 +41,26 @@ RUN python3 -m pip install --upgrade pip setuptools wheel html5lib six \ ...@@ -41,26 +41,26 @@ RUN python3 -m pip install --upgrade pip setuptools wheel html5lib six \
&& cd sglang \ && cd sglang \
&& if [ "$BUILD_TYPE" = "srt" ]; then \ && if [ "$BUILD_TYPE" = "srt" ]; then \
if [ "$CUDA_VERSION" = "12.1.1" ]; then \ if [ "$CUDA_VERSION" = "12.1.1" ]; then \
python3 -m pip --no-cache-dir install -e "python[srt]" --find-links https://flashinfer.ai/whl/cu121/torch2.4/flashinfer/; \ python3 -m pip --no-cache-dir install -e "python[srt]" --find-links https://flashinfer.ai/whl/cu121/torch2.5/flashinfer/; \
elif [ "$CUDA_VERSION" = "12.4.1" ]; then \ elif [ "$CUDA_VERSION" = "12.4.1" ]; then \
python3 -m pip --no-cache-dir install -e "python[srt]" --find-links https://flashinfer.ai/whl/cu124/torch2.4/flashinfer/; \ python3 -m pip --no-cache-dir install -e "python[srt]" --find-links https://flashinfer.ai/whl/cu124/torch2.5/flashinfer/; \
elif [ "$CUDA_VERSION" = "12.5.1" ]; then \ elif [ "$CUDA_VERSION" = "12.5.1" ]; then \
python3 -m pip --no-cache-dir install -e "python[srt]" --find-links https://flashinfer.ai/whl/cu124/torch2.4/flashinfer/; \ python3 -m pip --no-cache-dir install -e "python[srt]" --find-links https://flashinfer.ai/whl/cu124/torch2.5/flashinfer/; \
elif [ "$CUDA_VERSION" = "11.8.0" ]; then \ elif [ "$CUDA_VERSION" = "11.8.0" ]; then \
python3 -m pip --no-cache-dir install -e "python[srt]" --find-links https://flashinfer.ai/whl/cu118/torch2.4/flashinfer/; \ python3 -m pip --no-cache-dir install -e "python[srt]" --find-links https://flashinfer.ai/whl/cu118/torch2.5/flashinfer/; \
python3 -m pip install sgl-kernel -i https://docs.sglang.ai/whl/cu118; \ python3 -m pip install sgl-kernel -i https://docs.sglang.ai/whl/cu118; \
else \ else \
echo "Unsupported CUDA version: $CUDA_VERSION" && exit 1; \ echo "Unsupported CUDA version: $CUDA_VERSION" && exit 1; \
fi; \ fi; \
else \ else \
if [ "$CUDA_VERSION" = "12.1.1" ]; then \ if [ "$CUDA_VERSION" = "12.1.1" ]; then \
python3 -m pip --no-cache-dir install -e "python[all]" --find-links https://flashinfer.ai/whl/cu121/torch2.4/flashinfer/; \ python3 -m pip --no-cache-dir install -e "python[all]" --find-links https://flashinfer.ai/whl/cu121/torch2.5/flashinfer/; \
elif [ "$CUDA_VERSION" = "12.4.1" ]; then \ elif [ "$CUDA_VERSION" = "12.4.1" ]; then \
python3 -m pip --no-cache-dir install -e "python[all]" --find-links https://flashinfer.ai/whl/cu124/torch2.4/flashinfer/; \ python3 -m pip --no-cache-dir install -e "python[all]" --find-links https://flashinfer.ai/whl/cu124/torch2.5/flashinfer/; \
elif [ "$CUDA_VERSION" = "12.5.1" ]; then \ elif [ "$CUDA_VERSION" = "12.5.1" ]; then \
python3 -m pip --no-cache-dir install -e "python[all]" --find-links https://flashinfer.ai/whl/cu124/torch2.4/flashinfer/; \ python3 -m pip --no-cache-dir install -e "python[all]" --find-links https://flashinfer.ai/whl/cu124/torch2.5/flashinfer/; \
elif [ "$CUDA_VERSION" = "11.8.0" ]; then \ elif [ "$CUDA_VERSION" = "11.8.0" ]; then \
python3 -m pip --no-cache-dir install -e "python[all]" --find-links https://flashinfer.ai/whl/cu118/torch2.4/flashinfer/; \ python3 -m pip --no-cache-dir install -e "python[all]" --find-links https://flashinfer.ai/whl/cu118/torch2.5/flashinfer/; \
python3 -m pip install sgl-kernel -i https://docs.sglang.ai/whl/cu118; \ python3 -m pip install sgl-kernel -i https://docs.sglang.ai/whl/cu118; \
else \ else \
echo "Unsupported CUDA version: $CUDA_VERSION" && exit 1; \ echo "Unsupported CUDA version: $CUDA_VERSION" && exit 1; \
......
docs/start/install.md
View file @ 6186a8f8
...@@ -6,7 +6,7 @@ You can install SGLang using any of the methods below. ...@@ -6,7 +6,7 @@ You can install SGLang using any of the methods below.
``` ```
pip install --upgrade pip pip install --upgrade pip
pip install sgl-kernel --force-reinstall --no-deps pip install sgl-kernel --force-reinstall --no-deps
pip install "sglang[all]" --find-links https://flashinfer.ai/whl/cu124/torch2.4/flashinfer/ pip install "sglang[all]" --find-links https://flashinfer.ai/whl/cu124/torch2.5/flashinfer/
``` ```
Note: Please check the [FlashInfer installation doc](https://docs.flashinfer.ai/installation.html) to install the proper version according to your PyTorch and CUDA versions. Note: Please check the [FlashInfer installation doc](https://docs.flashinfer.ai/installation.html) to install the proper version according to your PyTorch and CUDA versions.
...@@ -19,7 +19,7 @@ cd sglang ...@@ -19,7 +19,7 @@ cd sglang
pip install --upgrade pip pip install --upgrade pip
pip install sgl-kernel --force-reinstall --no-deps pip install sgl-kernel --force-reinstall --no-deps
pip install -e "python[all]" --find-links https://flashinfer.ai/whl/cu124/torch2.4/flashinfer/ pip install -e "python[all]" --find-links https://flashinfer.ai/whl/cu124/torch2.5/flashinfer/
``` ```
Note: Please check the [FlashInfer installation doc](https://docs.flashinfer.ai/installation.html) to install the proper version according to your PyTorch and CUDA versions. If you meet with issue like **ImportError: cannot import name `_grouped_size_compiled_for_decode_kernels`**, installing FlashInfer with some older version like 0.1.6 instead of the latest version could solve it. Note: Please check the [FlashInfer installation doc](https://docs.flashinfer.ai/installation.html) to install the proper version according to your PyTorch and CUDA versions. If you meet with issue like **ImportError: cannot import name `_grouped_size_compiled_for_decode_kernels`**, installing FlashInfer with some older version like 0.1.6 instead of the latest version could solve it.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment