• Daniël de Kok's avatar
    nix: add default package (#2453) · f3c5d7d9
    Daniël de Kok authored
    The default package wraps the launcher and puts the server/router in the
    path.
    
    As a result, TGI can be started using something like:
    
    ```
    nix run .# -- \
      --model-id hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4 \
      --port 8080
    ```
    f3c5d7d9
flake.nix 2.96 KB