• Daniël de Kok's avatar
    nix: experimental support for building a Docker container (#2470) · 584b4d7a
    Daniël de Kok authored
    
    
    * nix: experimental support for building a Docker image
    
    Run using something like:
    
    ```
    docker run \
      --device nvidia.com/gpu=all \
      -it --rm -p 8080:80 \
      -v $PWD/data:/data \
      -v $PWD/tmp:/tmp \
      tgi-docker:latest \
      --model-id <model_id>
    ```
    
    * Example of building the Docker image using Nix inside Docker
    
    * Stream to make the builder image smaller
    
    This avoids storing a Docker image tarball in the image. Instead,
    stream the layers while doing `docker run`.
    
    * Don't spam journalctl on Linux
    
    * Other dockerfile.
    
    ---------
    Co-authored-by: default avatarNicolas Patry <patry.nicolas@protonmail.com>
    584b4d7a
flake.nix 4.62 KB