@@ -80,3 +80,30 @@ To port a model from vLLM to SGLang, you can compare these two files [SGLang Lla
- Remove `Sample`.
- Change `forward()` functions, and add `forward_batch`.
- Add `EntryClass` at the end.
### Registering an external model implementation
In addition to the methods described above, you can also register your new model with the `ModelRegistry` before launching the server. This approach is useful if you want to integrate your model without needing to modify the source code.
Here is how you can do it:
```python
fromsglang.srt.models.registryimportModelRegistry
fromsglang.srt.serverimportlaunch_server
# for a single model, you can add it to the registry
ModelRegistry.models[model_name]=model_class
# for multiple models, you can imitate the import_model_classes() function in sglang/srt/models/registry.py