• OlivierDehaene's avatar
    feat: Add token streaming using ServerSideEvents support (#36) · 7fbfbb0d
    OlivierDehaene authored
    Add token streaming using ServerSideEvents (SSE).
    
    The signature of the SSE events is: 
    
    ```rust
    struct Details {
        finish_reason: String,
        generated_tokens: u32,
        seed: Option<u64>,
    }
    
    struct StreamResponse {
        token: Token,
        generated_text: Option<String>,
        details: Option<Details>,
    }
    
    struct ErrorResponse {
        error: String,
    }
    ```
    7fbfbb0d
This project manages its dependencies using Cargo. Learn more
Cargo.toml 807 Bytes