"components/vscode:/vscode.git/clone" did not exist on "57728909cfb4005742b3bb0556a6c53a7047fa18"
  • Graham King's avatar
    docs: Example Chat sglang engine (#1015) · 24e2cbf5
    Graham King authored
    Example of how to connect a Python sglang engine to the message bus (NATS/etc). I
    
    In this example sglang does the pre/post processing. There is already an example where Dynamo does it.
    
    The examples teach this:
    
    - Be a chat completions engine, do your own pre-processing:
    
    ```
    await register_llm(ModelType.Chat, endpoint, config.model)
    ```
    
    - Have Dynamo do pre-processing. It will register us under both Chat and Completions endpoints, because that's handled before a Backend engine gets the request:
    
    ```
    await register_llm(ModelType.Backend, endpoint, config.model)
    ```
    24e2cbf5
dynamo_run.md 18.1 KB