Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
a33ea28b
Unverified
Commit
a33ea28b
authored
Jul 29, 2025
by
Michael Goin
Committed by
GitHub
Jul 29, 2025
Browse files
Add `flashinfer_python` to CUDA wheel requirements (#21389)
Signed-off-by:
mgoin
<
mgoin64@gmail.com
>
parent
7b49cb1c
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
5 additions
and
1 deletion
+5
-1
docker/Dockerfile
docker/Dockerfile
+3
-1
requirements/cuda.txt
requirements/cuda.txt
+2
-0
No files found.
docker/Dockerfile
View file @
a33ea28b
...
@@ -386,6 +386,8 @@ RUN --mount=type=bind,from=build,src=/workspace/dist,target=/vllm-workspace/dist
...
@@ -386,6 +386,8 @@ RUN --mount=type=bind,from=build,src=/workspace/dist,target=/vllm-workspace/dist
# Install FlashInfer from source
# Install FlashInfer from source
ARG
FLASHINFER_GIT_REPO="https://github.com/flashinfer-ai/flashinfer.git"
ARG
FLASHINFER_GIT_REPO="https://github.com/flashinfer-ai/flashinfer.git"
# Keep this in sync with https://github.com/vllm-project/vllm/blob/main/requirements/cuda.txt
# We use `--force-reinstall --no-deps` to avoid issues with the existing FlashInfer wheel.
ARG
FLASHINFER_GIT_REF="v0.2.9rc2"
ARG
FLASHINFER_GIT_REF="v0.2.9rc2"
RUN
--mount
=
type
=
cache,target
=
/root/.cache/uv bash -
<<
'
BASH
'
RUN
--mount
=
type
=
cache,target
=
/root/.cache/uv bash -
<<
'
BASH
'
. /etc/environment
. /etc/environment
...
@@ -408,7 +410,7 @@ RUN --mount=type=cache,target=/root/.cache/uv bash - <<'BASH'
...
@@ -408,7 +410,7 @@ RUN --mount=type=cache,target=/root/.cache/uv bash - <<'BASH'
TORCH_CUDA_ARCH_LIST="${FI_TORCH_CUDA_ARCH_LIST}" \
TORCH_CUDA_ARCH_LIST="${FI_TORCH_CUDA_ARCH_LIST}" \
python3 -m flashinfer.aot
python3 -m flashinfer.aot
TORCH_CUDA_ARCH_LIST="${FI_TORCH_CUDA_ARCH_LIST}" \
TORCH_CUDA_ARCH_LIST="${FI_TORCH_CUDA_ARCH_LIST}" \
uv pip install --system --no-build-isolation .
uv pip install --system --no-build-isolation
--force-reinstall --no-deps
.
popd
popd
rm -rf flashinfer
rm -rf flashinfer
BASH
BASH
...
...
requirements/cuda.txt
View file @
a33ea28b
...
@@ -12,3 +12,5 @@ torchaudio==2.7.1
...
@@ -12,3 +12,5 @@ torchaudio==2.7.1
torchvision==0.22.1 # Required for phi3v processor. See https://github.com/pytorch/vision?tab=readme-ov-file#installation for corresponding version
torchvision==0.22.1 # Required for phi3v processor. See https://github.com/pytorch/vision?tab=readme-ov-file#installation for corresponding version
# https://github.com/facebookresearch/xformers/releases/tag/v0.0.31
# https://github.com/facebookresearch/xformers/releases/tag/v0.0.31
xformers==0.0.31; platform_system == 'Linux' and platform_machine == 'x86_64' # Requires PyTorch >= 2.7
xformers==0.0.31; platform_system == 'Linux' and platform_machine == 'x86_64' # Requires PyTorch >= 2.7
# FlashInfer should be updated together with the Dockerfile
flashinfer_python==0.2.9rc2
\ No newline at end of file
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment