Unverified Commit f3c5d7d9 authored by Daniël de Kok's avatar Daniël de Kok Committed by GitHub
Browse files

nix: add default package (#2453)

The default package wraps the launcher and puts the server/router in the
path.

As a result, TGI can be started using something like:

```
nix run .# -- \
  --model-id hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4 \
  --port 8080
```
parent 358ceb67
...@@ -99,6 +99,17 @@ ...@@ -99,6 +99,17 @@
''; '';
}; };
}; };
packages.default = pkgs.writeShellApplication {
name = "text-generation-inference";
runtimeInputs = [
server
router
];
text = ''
${launcher}/bin/text-generation-launcher "$@"
'';
};
} }
); );
} }
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment