• Graham King's avatar
    feat(dynamo-run): Download models from HF, smart model defaults (#126) · 089f8e1b
    Graham King authored
    
    
    - Any engine can take the name of a Hugging Face repository. It will be downloaded before calling the engine.
    
    - The default engine (previously always mistralrs) depends on what is compiled in.
    
    - Text can be piped in and will result in a single run of the model.
    
    All of those together mean if you build with `--features vllm` you can do this and it will download the model and run it with vllm, answer your question, and exit:
    ```
    echo "What is the capital of Costa Rica?"  | dynamo-run Qwen/Qwen2.5-3B-Instruct
    ```
    Co-authored-by: default avatarRyan McCormick <rmccormick@nvidia.com>
    089f8e1b
README.md 13.2 KB