Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
8354f664
Unverified
Commit
8354f664
authored
Feb 22, 2025
by
Cyrus Leung
Committed by
GitHub
Feb 22, 2025
Browse files
[Doc] Dockerfile instructions for optional dependencies and dev transformers (#13699)
parent
c904fddd
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
30 additions
and
0 deletions
+30
-0
docs/source/deployment/docker.md
docs/source/deployment/docker.md
+30
-0
No files found.
docs/source/deployment/docker.md
View file @
8354f664
...
...
@@ -27,6 +27,36 @@ container to access the host's shared memory. vLLM uses PyTorch, which uses shar
memory to share data between processes under the hood, particularly for tensor parallel inference.
:::
:::{note}
Optional dependencies are not included in order to avoid licensing issues (e.g.
<gh-issue:8030>
).
If you need to use those dependencies (having accepted the license terms),
create a custom Dockerfile on top of the base image with an extra layer that installs them:
```
Dockerfile
FROM
vllm/vllm-openai:v0.7.3
# e.g. install the `audio` and `video` optional dependencies
# NOTE: Make sure the version of vLLM matches the base image!
RUN
uv pip
install
--system
vllm[audio,video]
==
0.7.3
```
:::
:::{tip}
Some new models may only be available on the main branch of
[
HF Transformers
](
https://github.com/huggingface/transformers
)
.
To use the development version of
`transformers`
, create a custom Dockerfile on top of the base image
with an extra layer that installs their code from source:
```
Dockerfile
FROM
vllm/vllm-openai:latest
RUN
uv pip
install
--system
git+https://github.com/huggingface/transformers.git
```
:::
(deployment-docker-build-image-from-source)=
## Building vLLM's Docker Image from Source
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment