GitLab

Co-authored-by: Bruce MacDonald <brucewmacdonald@gmail.com>

Separate GGUF and PyTorch guides

Signed-off-by: Matt Williams <m@technovangelist.com>

Signed-off-by: Matt Williams <m@technovangelist.com>

Signed-off-by: Matt Williams <m@technovangelist.com>

* update streaming request accept header
* add optional stream param to request bodies

Avoid triple ticks in visual editor and also copied in clipboard.

Document response stream chunk delimiter.

* enable packaging multiple cuda versions
* use nvcc cuda version if available

---------
Co-authored-by: Michael Yang <mxyng@pm.me>

* Update API docs
Signed-off-by: Matt Williams <m@technovangelist.com>

* strange TOC was getting auto generated
Signed-off-by: Matt Williams <m@technovangelist.com>

* Update docs/api.md
Co-authored-by: Bruce MacDonald <brucewmacdonald@gmail.com>

* Update docs/api.md
Co-authored-by: Bruce MacDonald <brucewmacdonald@gmail.com>

* Update docs/api.md
Co-authored-by: Bruce MacDonald <brucewmacdonald@gmail.com>

* Update api.md

---------
Signed-off-by: Matt Williams <m@technovangelist.com>
Co-authored-by: Bruce MacDonald <brucewmacdonald@gmail.com>
Co-authored-by: Michael Chiang <mchiang0610@users.noreply.github.com>

* linux gpu support
* handle multiple gpus
* add cuda docker image (#488)
---------
Co-authored-by: Michael Yang <mxyng@pm.me>

* remove c code
* pack llama.cpp
* use request context for llama_cpp
* let llama_cpp decide the number of threads to use
* stop llama runner when app stops
* remove sample count and duration metrics
* use go generate to get libraries
* tmp dir for running llm