Unverified Commit c5d7fb9d authored by Russell Bryant's avatar Russell Bryant Committed by GitHub
Browse files

[Doc] fix third-party model example (#9771)


Signed-off-by: default avatarRussell Bryant <rbryant@redhat.com>
parent 76ed5340
...@@ -133,7 +133,9 @@ If you are running api server with :code:`vllm serve <args>`, you can wrap the e ...@@ -133,7 +133,9 @@ If you are running api server with :code:`vllm serve <args>`, you can wrap the e
from vllm import ModelRegistry from vllm import ModelRegistry
from your_code import YourModelForCausalLM from your_code import YourModelForCausalLM
ModelRegistry.register_model("YourModelForCausalLM", YourModelForCausalLM) ModelRegistry.register_model("YourModelForCausalLM", YourModelForCausalLM)
import runpy
runpy.run_module('vllm.entrypoints.openai.api_server', run_name='__main__') if __name__ == '__main__':
import runpy
runpy.run_module('vllm.entrypoints.openai.api_server', run_name='__main__')
Save the above code in a file and run it with :code:`python your_file.py <args>`. Save the above code in a file and run it with :code:`python your_file.py <args>`.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment