README.md 11.3 KB
Newer Older
1
# Contributing to vLLM
2
3
4
5
6
7
8
9
10
11
12
13

Thank you for your interest in contributing to vLLM! Our community is open to everyone and welcomes all kinds of contributions, no matter how small or large. There are several ways you can contribute to the project:

- Identify and report any issues or bugs.
- Request or add support for a new model.
- Suggest or implement new features.
- Improve documentation or contribute a how-to guide.

We also believe in the power of community support; thus, answering queries, offering PR reviews, and assisting others are also highly regarded and beneficial contributions.

Finally, one of the most impactful ways to support us is by raising awareness about vLLM. Talk about it in your blog posts and highlight how it's driving your incredible projects. Express your support on social media if you're using vLLM, or simply offer your appreciation by starring our repository!

14
15
16
17
18
## Job Board

Unsure on where to start? Check out the following links for tasks to work on:

- [Good first issues](https://github.com/vllm-project/vllm/issues?q=is%3Aissue%20state%3Aopen%20label%3A%22good%20first%20issue%22)
19
    - [Selected onboarding tasks](gh-project:6)
20
- [New model requests](https://github.com/vllm-project/vllm/issues?q=is%3Aissue%20state%3Aopen%20label%3A%22new-model%22)
21
    - [Models with multi-modal capabilities](gh-project:10)
22

23
## License
24

25
See <gh-file:LICENSE>.
26

27
## Developing
28

29
30
--8<-- "docs/getting_started/installation/python_env_setup.inc.md"

31
Depending on the kind of development you'd like to do (e.g. Python, CUDA), you can choose to build vLLM with or without compilation.
32
33
Check out the [building from source][build-from-source] documentation for details.

34
35
For an optimized workflow when iterating on C++/CUDA kernels, see the [Incremental Compilation Workflow](./incremental_build.md) for recommendations.

Reid's avatar
Reid committed
36
### Building the docs with MkDocs
37

Reid's avatar
Reid committed
38
39
40
41
42
43
44
#### Introduction to MkDocs

[MkDocs](https://github.com/mkdocs/mkdocs) is a fast, simple and downright gorgeous static site generator that's geared towards building project documentation. Documentation source files are written in Markdown, and configured with a single YAML configuration file.

#### Install MkDocs and Plugins

Install MkDocs along with the [plugins](https://github.com/vllm-project/vllm/blob/main/mkdocs.yaml) used in the vLLM documentation, as well as required dependencies:
45
46

```bash
47
uv pip install -r requirements/docs.txt
48
49
```

Reid's avatar
Reid committed
50
51
!!! note
    Ensure that your Python version is compatible with the plugins (e.g., `mkdocs-awesome-nav` requires Python 3.10+)
Reid's avatar
Reid committed
52
53
54

#### Verify Installation

Reid's avatar
Reid committed
55
Confirm that MkDocs is correctly installed:
Reid's avatar
Reid committed
56
57
58
59
60
61
62
63

```bash
mkdocs --version
```

Example output:

```console
Reid's avatar
Reid committed
64
mkdocs, version 1.6.1 from /opt/miniconda3/envs/mkdoc/lib/python3.10/site-packages/mkdocs (Python 3.10)
Reid's avatar
Reid committed
65
66
67
68
69
70
71
72
73
74
75
76
```

#### Clone the `vLLM` repository

```bash
git clone https://github.com/vllm-project/vllm.git
cd vllm
```

#### Start the Development Server

MkDocs comes with a built-in dev-server that lets you preview your documentation as you work on it. Make sure you're in the same directory as the `mkdocs.yml` configuration file, and then start the server by running the `mkdocs serve` command:
77
78
79
80

```bash
mkdocs serve
```
81

Reid's avatar
Reid committed
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
Example output:

```console
INFO    -  Documentation built in 106.83 seconds
INFO    -  [22:02:02] Watching paths for changes: 'docs', 'mkdocs.yaml'
INFO    -  [22:02:02] Serving on http://127.0.0.1:8000/
```

#### View in Your Browser

Open up [http://127.0.0.1:8000/](http://127.0.0.1:8000/) in your browser to see a live preview:.

#### Learn More

For additional features and advanced configurations, refer to the official [MkDocs Documentation](https://www.mkdocs.org/).

98
## Testing
99

100
??? console "Commands"
101

102
    ```bash
103
104
    # These commands are only for Nvidia CUDA platforms.
    uv pip install -r requirements/common.txt -r requirements/dev.txt --torch-backend=auto
105

106
    # Linting, formatting and static type checking
107
    pre-commit install
108

109
    # You can manually run pre-commit with
110
    pre-commit run --all-files --show-diff-on-failure
111

112
113
114
    # To manually run something from CI that does not run
    # locally by default, you can run:
    pre-commit run mypy-3.9 --hook-stage manual --all-files
Reid's avatar
Reid committed
115

116
117
118
119
120
121
    # Unit tests
    pytest tests/

    # Run tests for a single test file with detailed output
    pytest -s -v tests/test_logger.py
    ```
122

123
124
!!! tip
    Since the <gh-file:docker/Dockerfile> ships with Python 3.12, all tests in CI (except `mypy`) are run with Python 3.12.
125

126
    Therefore, we recommend developing with Python 3.12 to minimise the chance of your local environment clashing with our CI environment.
127

128
129
130
131
!!! note "Install python3-dev if Python.h is missing"
    If any of the above commands fails with `Python.h: No such file or directory`, install
    `python3-dev` with `sudo apt install python3-dev`.

132
133
!!! note
    Currently, the repository is not fully checked by `mypy`.
134

135
136
137
138
!!! note
    Currently, not all unit tests pass when run on CPU platforms. If you don't have access to a GPU
    platform to run unit tests locally, rely on the continuous integration system to run the tests for
    now.
139

140
## Issues
141

142
If you encounter a bug or have a feature request, please [search existing issues](https://github.com/vllm-project/vllm/issues?q=is%3Aissue) first to see if it has already been reported. If not, please [file a new issue](https://github.com/vllm-project/vllm/issues/new/choose), providing as much relevant information as possible.
143

144
!!! important
145
    If you discover a security vulnerability, please follow the instructions [here](gh-file:SECURITY.md#reporting-a-vulnerability).
146

147
## Pull Requests & Code Reviews
148

149
150
151
152
Thank you for your contribution to vLLM! Before submitting the pull request,
please ensure the PR meets the following criteria. This helps vLLM maintain the
code quality and improve the efficiency of the review process.

153
### DCO and Signed-off-by
154

155
When contributing changes to this project, you must agree to the <gh-file:DCO>.
156
Commits must include a `Signed-off-by:` header which certifies agreement with
157
the terms of the DCO.
158

159
Using `-s` with `git commit` will automatically add this header.
160

161
!!! tip
162
    You can enable automatic sign-off via your IDE:
163

164
165
166
167
    - **PyCharm**: Click on the `Show Commit Options` icon to the right of the `Commit and Push...` button in the `Commit` window.
      It will bring up a `git` window where you can modify the `Author` and enable `Sign-off commit`.
    - **VSCode**: Open the [Settings editor](https://code.visualstudio.com/docs/configure/settings)
      and enable the `Git: Always Sign Off` (`git.alwaysSignOff`) field.
168

169
### PR Title and Classification
170
171
172
173

Only specific types of PRs will be reviewed. The PR title is prefixed
appropriately to indicate the type of change. Please use one of the following:

174
175
176
177
- `[Bugfix]` for bug fixes.
- `[CI/Build]` for build or continuous integration improvements.
- `[Doc]` for documentation fixes and improvements.
- `[Model]` for adding a new model or improving an existing model. Model name
178
  should appear in the title.
179
180
181
182
183
184
185
186
- `[Frontend]` For changes on the vLLM frontend (e.g., OpenAI API server,
  `LLM` class, etc.)
- `[Kernel]` for changes affecting CUDA kernels or other compute kernels.
- `[Core]` for changes in the core vLLM logic (e.g., `LLMEngine`,
  `AsyncLLMEngine`, `Scheduler`, etc.)
- `[Hardware][Vendor]` for hardware-specific changes. Vendor name should
  appear in the prefix (e.g., `[Hardware][AMD]`).
- `[Misc]` for PRs that do not fit the above categories. Please use this
187
188
  sparingly.

189
190
!!! note
    If the PR spans more than one category, please include all relevant prefixes.
191

192
### Code Quality
193
194
195

The PR needs to meet the following code quality standards:

196
- We adhere to [Google Python style guide](https://google.github.io/styleguide/pyguide.html) and [Google C++ style guide](https://google.github.io/styleguide/cppguide.html).
197
198
- Pass all linter checks. Please use `pre-commit` to format your code. See
  <https://pre-commit.com/#usage> if `pre-commit` is new to you.
199
200
201
202
- The code needs to be well-documented to ensure future contributors can easily
  understand the code.
- Include sufficient tests to ensure the project stays correct and robust. This
  includes both unit tests and integration tests.
203
204
- Please add documentation to `docs/` if the PR modifies the user-facing behaviors of vLLM.
  It helps vLLM users understand and utilize the new features or changes.
205

206
### Adding or Changing Kernels
207

208
When actively developing or modifying kernels, using the [Incremental Compilation Workflow](./incremental_build.md) is highly recommended for faster build times.
209
210
211
Each custom kernel needs a schema and one or more implementations to be registered with PyTorch.

- Make sure custom ops are registered following PyTorch guidelines:
212
213
214
  [Custom C++ and CUDA Operators](https://pytorch.org/tutorials/advanced/cpp_custom_ops.html#cpp-custom-ops-tutorial)
  and [The Custom Operators Manual](https://docs.google.com/document/d/1_W62p8WJOQQUzPsJYa7s701JXt0qf2OfLub2sbkHOaU).
- Custom operations that return `Tensors` require meta-functions.
215
216
217
  Meta-functions should be implemented and registered in Python so that dynamic
  dims can be handled automatically. See above documents for a description of
  meta-functions.
218
- Use [torch.library.opcheck()](https://pytorch.org/docs/stable/library.html#torch.library.opcheck)
219
  to test the function registration and meta-function for any registered ops.
220
  See `tests/kernels` for examples.
221
222
223
- When changing the C++ signature of an existing op, the schema must be updated
  to reflect the changes.
- If a new custom type is needed, see the following document:
224
  [Custom Class Support in PT2](https://docs.google.com/document/d/18fBMPuOJ0fY5ZQ6YyrHUppw9FA332CpNtgB6SOIgyuA).
225

226
### Notes for Large Changes
227
228
229
230

Please keep the changes as concise as possible. For major architectural changes
(>500 LOC excluding kernel/data/config/test), we would expect a GitHub issue
(RFC) discussing the technical design and justification. Otherwise, we will tag
231
it with `rfc-required` and might not go through the PR.
232

233
### What to Expect for the Reviews
234
235
236
237
238
239
240
241
242
243
244
245

The goal of the vLLM team is to be a *transparent reviewing machine*. We would
like to make the review process transparent and efficient and make sure no
contributor feels confused or frustrated. However, the vLLM team is small, so we
need to prioritize some PRs over others. Here is what you can expect from the
review process:

- After the PR is submitted, the PR will be assigned to a reviewer. Every
  reviewer will pick up the PRs based on their expertise and availability.
- After the PR is assigned, the reviewer will provide status updates every 2-3
  days. If the PR is not reviewed within 7 days, please feel free to ping the
  reviewer or the vLLM team.
246
- After the review, the reviewer will put an `action-required` label on the PR
247
248
249
250
251
  if there are changes required. The contributor should address the comments and
  ping the reviewer to re-review the PR.
- Please respond to all comments within a reasonable time frame. If a comment
  isn't clear or you disagree with a suggestion, feel free to ask for
  clarification or discuss the suggestion.
252
253
254
- Note that not all CI checks will be executed due to limited computational
  resources. The reviewer will add `ready` label to the PR when the PR is
  ready to merge or a full CI run is needed.
255

256
## Thank You
257
258
259

Finally, thank you for taking the time to read these guidelines and for your interest in contributing to vLLM.
All of your contributions help make vLLM a great tool and community for everyone!