Unverified Commit c5d7fb9d authored by Russell Bryant's avatar Russell Bryant Committed by GitHub
Browse files

[Doc] fix third-party model example (#9771)


Signed-off-by: default avatarRussell Bryant <rbryant@redhat.com>
parent 76ed5340
......@@ -133,6 +133,8 @@ If you are running api server with :code:`vllm serve <args>`, you can wrap the e
from vllm import ModelRegistry
from your_code import YourModelForCausalLM
ModelRegistry.register_model("YourModelForCausalLM", YourModelForCausalLM)
if __name__ == '__main__':
import runpy
runpy.run_module('vllm.entrypoints.openai.api_server', run_name='__main__')
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment