• Graham King's avatar
    chore(dynamo-run): Refactor to library (#1687) · 92f06b0e
    Graham King authored
    Move much of what was in the `dynamo-run` crate into `dynamo-llm` so that everyone can use it.
    
    Example usage:
    
    1. Create a `LocalModel`:
    
    ```
        let local_model = LocalModelBuilder::default()
    	.model_path("Qwen/Qwen3-0.6B")
    	.http_port(8080)
    	.build().await?;
    ```
    
    2. Make an engine:
    
    ```
        let engine_config = EngineConfig::StaticFull {
    	engine: dynamo_engine_mistralrs::make_engine(&local_model).await?,
    	model: Box::new(local_model),
        };
    ```
    
    3. Connect it to an input and run it
    
    ```
        dynamo_llm::entrypoint::input::run_input(Input::Http, runtime, engine_config).await?;
    ```
    
    For https://github.com/ai-dynamo/dynamo/issues/1647
    
    Code Rabbit summary, thanks:
      * Introduced a flexible builder pattern for local model configuration, allowing advanced customization and easier initialization.
      * Added new input modes and unified input handling, supporting interactive chat, HTTP server, batch file, and distributed endpoint modes.
      * Centralized engine configuration and routing, enabling more extensible and maintainable engine management.
      * Simplified and modularized the codebase by moving input and engine logic into dedicated modules.
      * Replaced direct model construction with an asynchronous builder for improved clarity and extensibility.
      * Streamlined configuration and validation for flags and router settings.
      * Added validation to prevent incompatible input and output combinations in endpoint and dynamic modes.
    92f06b0e
sp.rs 3.62 KB