Skip to content

GitLab

Unverified Commit c5d7fb9d authored by Russell Bryant's avatar Russell Bryant Committed by GitHub
Browse files

[Doc] fix third-party model example (#9771) 


Signed-off-by: default avatarRussell Bryant <rbryant@redhat.com>
parent 76ed5340
Show whitespace changes
Inline Side-by-side
Showing with 4 additions and 2 deletions
docs/source/models/adding_model.rst
View file @ c5d7fb9d
...@@ -133,6 +133,8 @@ If you are running api server with :code:`vllm serve <args>`, you can wrap the e ...@@ -133,6 +133,8 @@ If you are running api server with :code:`vllm serve <args>`, you can wrap the e
from vllm import ModelRegistry from vllm import ModelRegistry
from your_code import YourModelForCausalLM from your_code import YourModelForCausalLM
ModelRegistry.register_model("YourModelForCausalLM", YourModelForCausalLM) ModelRegistry.register_model("YourModelForCausalLM", YourModelForCausalLM)
if __name__ == '__main__':
import runpy import runpy
runpy.run_module('vllm.entrypoints.openai.api_server', run_name='__main__') runpy.run_module('vllm.entrypoints.openai.api_server', run_name='__main__')
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment