• Graham King's avatar
    feat(dynamo-run): Basic routing choice (#524) · ec2e7307
    Graham King authored
    As a first step towards KV routing:
    - introduce a `--router-mode` in dynamo-run that only does random and round-robin right now. Not that interesting yet.
    - Make the vllm engine publish the KV events received from our patched vllm.
    
    Now we "just" need to connect the two. Easy right?
    ec2e7307
common.rs 3.96 KB