Unverified Commit 1aa0fbf4 authored by Brayden Zhong's avatar Brayden Zhong Committed by GitHub
Browse files

Add note to add supported model to documentation (#6640)

parent 7a0bbe6a
...@@ -15,11 +15,13 @@ python3 -m sglang.launch_server \ ...@@ -15,11 +15,13 @@ python3 -m sglang.launch_server \
Below the supported models are summarized in a table. Below the supported models are summarized in a table.
If unsure if a specific architecture is implemented you may look it up via github search for example to search for `Qwen3ForCausalLm` use the expression If you are unsure if a specific architecture is implemented, you can search for it via GitHub. For example, to search for `Qwen3ForCausalLM`, use the expression:
``` ```
repo:sgl-project/sglang path:/^python\/sglang\/srt\/models\// Qwen3ForCausalLm repo:sgl-project/sglang path:/^python\/sglang\/srt\/models\// Qwen3ForCausalLM
``` ```
in Github search.
in the GitHub search bar.
| Model Family (Variants) | Example HuggingFace Identifier | Description | | Model Family (Variants) | Example HuggingFace Identifier | Description |
|-------------------------------------|--------------------------------------------------|----------------------------------------------------------------------------------------| |-------------------------------------|--------------------------------------------------|----------------------------------------------------------------------------------------|
......
...@@ -15,11 +15,13 @@ python3 -m sglang.launch_server \ ...@@ -15,11 +15,13 @@ python3 -m sglang.launch_server \
Below the supported models are summarized in a table. Below the supported models are summarized in a table.
If unsure if a specific architecture is implemented you may look it up via Github search. For example to search for `Qwen2_5_VLForConditionalGeneration` use the expression If you are unsure if a specific architecture is implemented, you can search for it via GitHub. For example, to search for `Qwen2_5_VLForConditionalGeneration`, use the expression:
``` ```
repo:sgl-project/sglang path:/^python\/sglang\/srt\/models\// Qwen2_5_VLForConditionalGeneration repo:sgl-project/sglang path:/^python\/sglang\/srt\/models\// Qwen2_5_VLForConditionalGeneration
``` ```
in Github search.
in the GitHub search bar.
| Model Family (Variants) | Example HuggingFace Identifier | Chat Template | Description | | Model Family (Variants) | Example HuggingFace Identifier | Chat Template | Description |
......
# How to Support New Models # How to Support New Models
This document explains how to add support for new language models and multimodal large language models (mllms) in This document explains how to add support for new language models and multimodal large language models (MLLMs) in
SGLang. It also covers how to test new models and register external implementations. SGLang. It also covers how to test new models and register external implementations.
## How to Support a new Language Model ## How to Support a New Language Model
To support a new model in SGLang, you only need to add a single file under To support a new model in SGLang, you only need to add a single file under
the [SGLang Models Directory](https://github.com/sgl-project/sglang/tree/main/python/sglang/srt/models). You can learn the [SGLang Models Directory](https://github.com/sgl-project/sglang/tree/main/python/sglang/srt/models). You can learn
...@@ -11,7 +11,7 @@ from existing model implementations and create a new file for your model. For mo ...@@ -11,7 +11,7 @@ from existing model implementations and create a new file for your model. For mo
similar model to start with (e.g., starting from Llama). Also refer how similar model to start with (e.g., starting from Llama). Also refer how
to [port a Model from vLLM to SGLang](#port-a-model-from-vllm-to-sglang) to [port a Model from vLLM to SGLang](#port-a-model-from-vllm-to-sglang)
## How to Support a new Multimodal Large Language Model ## How to Support a New Multimodal Large Language Model
To support a new multimodal large language model (MLLM) in SGLang, there are several key components in addition to the To support a new multimodal large language model (MLLM) in SGLang, there are several key components in addition to the
standard LLM support: standard LLM support:
...@@ -94,6 +94,8 @@ To port a model from vLLM to SGLang: ...@@ -94,6 +94,8 @@ To port a model from vLLM to SGLang:
- **Add `EntryClass`** at the end. - **Add `EntryClass`** at the end.
- **Ensure that the new implementation uses only SGLang components** and does not rely on any vLLM components. - **Ensure that the new implementation uses only SGLang components** and does not rely on any vLLM components.
Note: make sure you add your new model to the supported models list in the supported models documentation.
## Registering an External Model Implementation ## Registering an External Model Implementation
In addition to the methods above, you can register your new model with the `ModelRegistry` before launching the server. In addition to the methods above, you can register your new model with the `ModelRegistry` before launching the server.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment