Skip to content

GitLab

Unverified Commit f0ca16f0 authored by jthomson04's avatar jthomson04 Committed by GitHub
Browse files

feat: Better cublas support in vLLM container. (#4514) 


Signed-off-by: default avatarjthomson04 <jwillthomson19@gmail.com>
parent c0e394d5
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
container/Dockerfile.vllm
View file @ f0ca16f0
......@@ -217,8 +217,8 @@ COPY --from=framework /usr/local/cuda/bin/fatbinary /usr/local/cuda/bin/fatbinar
COPY --from=framework /usr/local/cuda/include/ /usr/local/cuda/include/
COPY --from=framework /usr/local/cuda/nvvm /usr/local/cuda/nvvm
COPY --from=framework /usr/local/cuda/lib64/libcudart.so* /usr/local/cuda/lib64/
COPY --from=framework /usr/local/cuda/lib64/libcublas.so* /usr/local/cuda/lib64/
COPY --from=framework /usr/local/cuda/lib64/libcublasLt.so* /usr/local/cuda/lib64/
RUN ln -s /usr/local/cuda/lib64/libcublas.so.12 /usr/local/cuda/lib64/libcublas.so
RUN ln -s /usr/local/cuda/lib64/libcublasLt.so.12 /usr/local/cuda/lib64/libcublasLt.so
### COPY NATS & ETCD ###
# Copy nats and etcd from dev image
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment