"components/vscode:/vscode.git/clone" did not exist on "57728909cfb4005742b3bb0556a6c53a7047fa18"
-
Graham King authored
Example of how to connect a Python sglang engine to the message bus (NATS/etc). I In this example sglang does the pre/post processing. There is already an example where Dynamo does it. The examples teach this: - Be a chat completions engine, do your own pre-processing: ``` await register_llm(ModelType.Chat, endpoint, config.model) ``` - Have Dynamo do pre-processing. It will register us under both Chat and Completions endpoints, because that's handled before a Backend engine gets the request: ``` await register_llm(ModelType.Backend, endpoint, config.model) ```
24e2cbf5