Unverified Commit 81a90e52 authored by Nicolò Lucchesi's avatar Nicolò Lucchesi Committed by GitHub
Browse files

[Docs] Add bart-plugin to docs (#33905)


Signed-off-by: default avatarNickLucche <nlucches@redhat.com>
parent 1c3a221d
...@@ -45,7 +45,7 @@ Every plugin has three parts: ...@@ -45,7 +45,7 @@ Every plugin has three parts:
## Types of supported plugins ## Types of supported plugins
- **General plugins** (with group name `vllm.general_plugins`): The primary use case for these plugins is to register custom, out-of-the-tree models into vLLM. This is done by calling `ModelRegistry.register_model` to register the model inside the plugin function. - **General plugins** (with group name `vllm.general_plugins`): The primary use case for these plugins is to register custom, out-of-the-tree models into vLLM. This is done by calling `ModelRegistry.register_model` to register the model inside the plugin function. For an example of an official model plugin, see the [bart-plugin](https://github.com/vllm-project/bart-plugin) which adds support for `BartForConditionalGeneration`.
- **Platform plugins** (with group name `vllm.platform_plugins`): The primary use case for these plugins is to register custom, out-of-the-tree platforms into vLLM. The plugin function should return `None` when the platform is not supported in the current environment, or the platform class's fully qualified name when the platform is supported. - **Platform plugins** (with group name `vllm.platform_plugins`): The primary use case for these plugins is to register custom, out-of-the-tree platforms into vLLM. The plugin function should return `None` when the platform is not supported in the current environment, or the platform class's fully qualified name when the platform is supported.
......
...@@ -174,6 +174,16 @@ class MyConfig(PretrainedConfig): ...@@ -174,6 +174,16 @@ class MyConfig(PretrainedConfig):
- The `list` in the first element of the `tuple` contains the names of the input arguments - The `list` in the first element of the `tuple` contains the names of the input arguments
- The `list` in the last element of the `tuple` contains the names of the variables the layer outputs to in your modeling code - The `list` in the last element of the `tuple` contains the names of the variables the layer outputs to in your modeling code
### Plugins
Some model architectures are supported via vLLM plugins. These plugins extend vLLM's capabilities through the [plugin system](../design/plugin_system.md).
| Architecture | Models | Plugin Repository |
|--------------|--------|-------------------|
| `BartForConditionalGeneration` | BART | [bart-plugin](https://github.com/vllm-project/bart-plugin) |
For other model architectures not natively supported, in particular for Encoder-Decoder models, we recommend following a similar pattern by implementing support through the plugin system.
## Loading a Model ## Loading a Model
### Hugging Face Hub ### Hugging Face Hub
......
...@@ -134,9 +134,12 @@ Please note that prefix caching is not yet supported for any of the above models ...@@ -134,9 +134,12 @@ Please note that prefix caching is not yet supported for any of the above models
#### Encoder-Decoder Models #### Encoder-Decoder Models
Whisper is supported. Other models requiring cross-attention between separate Whisper is supported natively. Other encoder-decoder models are supported via the plugin system:
encoder and decoder (e.g., `BartForConditionalGeneration`,
`MllamaForConditionalGeneration`) are no longer supported. - **BART**: `BartForConditionalGeneration` is supported via the official [bart-plugin](https://github.com/vllm-project/bart-plugin).
For other encoder-decoder models (e.g., `MllamaForConditionalGeneration`), we recommend
following a similar pattern by implementing support through the [plugin system](../design/plugin_system.md).
### Features ### Features
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment