Skip to content

GitLab

Unverified Commit 53cec4ac authored by Guangtong Bai's avatar Guangtong Bai Committed by GitHub
Browse files

feat: install Run:ai model streamer for vllm (#4848) 


Signed-off-by: default avatarGuangtong Bai <guangtong.bai@gmail.com>
parent 2ef0f5fb
Show whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
container/deps/vllm/install_vllm.sh
View file @ 53cec4ac
...@@ -126,7 +126,7 @@ git checkout $VLLM_REF ...@@ -126,7 +126,7 @@ git checkout $VLLM_REF
echo "\n=== Installing vLLM & FlashInfer ===" echo "\n=== Installing vLLM & FlashInfer ==="
echo "Installing vLLM $VLLM_REF from PyPI..." echo "Installing vLLM $VLLM_REF from PyPI..."
uv pip install vllm[flashinfer]==$VLLM_REF --torch-backend=${TORCH_BACKEND} uv pip install vllm[flashinfer,runai]==$VLLM_REF --torch-backend=${TORCH_BACKEND}
uv pip install flashinfer-cubin==$FLASHINF_REF uv pip install flashinfer-cubin==$FLASHINF_REF
uv pip install flashinfer-jit-cache==$FLASHINF_REF --extra-index-url https://flashinfer.ai/whl/${TORCH_BACKEND} uv pip install flashinfer-jit-cache==$FLASHINF_REF --extra-index-url https://flashinfer.ai/whl/${TORCH_BACKEND}
......
pyproject.toml
View file @ 53cec4ac
...@@ -56,7 +56,7 @@ trtllm =[ ...@@ -56,7 +56,7 @@ trtllm =[
vllm = [ vllm = [
"uvloop", "uvloop",
"nixl[cu12]<=0.7.1", "nixl[cu12]<=0.7.1",
"vllm[flashinfer]==0.12.0", "vllm[flashinfer,runai]==0.12.0",
] ]
sglang = [ sglang = [
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment