• Graham King's avatar
    feat(dynamo-run): Various UX improvements (#168) · 1fb31d6a
    Graham King authored
    Engines mistralrs, sglang and vllm included by default. Can be disabled like this: `cargo build --no-default-features --features <add-back-what-you-want>`.
    
    Added `--feature vulkan` option, for llamacpp.
    
    Build time message if CUDA or Metal would help and are missing. That's the best we can do:
    > warning: dynamo-run@0.1.0: CUDA not enabled, re-run with `--features cuda`
    
    Runtime message if CUDA, Metal or Vulkan are enabled:
    > 2025-03-14T21:59:26.501937Z  INFO dynamo_run: CUDA on
    
    Runtime message if they are missing:
    > 2025-03-14T22:02:37.439404Z  INFO dynamo_run: CPU mode. Rebuild with `--features cuda|metal|vulkan` for better performance
    
    Defaut engine message includes available engines:
    > 2025-03-14T21:59:26.503612Z  INFO dynamo_run: Using default engine: mistralrs. Use out=<engine> to specify one of echo_core, echo_full, mistralrs, llamacpp, sglang, vllm, pystr, pytok
    
    The really important outcome is that this should now "just work":
    ```
    cargo install dynamo-run
    dynamo-run Qwen/Qwen2.5-3B-Instruct
    ```
    
    Sadly you still need `--features cuda|metal` for performance, I couldn't automate that.
    1fb31d6a
main.rs 7.71 KB