GitLab

Document response stream chunk delimiter.

* enable packaging multiple cuda versions
* use nvcc cuda version if available

---------
Co-authored-by: Michael Yang <mxyng@pm.me>

* Update API docs
Signed-off-by: Matt Williams <m@technovangelist.com>

* strange TOC was getting auto generated
Signed-off-by: Matt Williams <m@technovangelist.com>

* Update docs/api.md
Co-authored-by: Bruce MacDonald <brucewmacdonald@gmail.com>

* Update docs/api.md
Co-authored-by: Bruce MacDonald <brucewmacdonald@gmail.com>

* Update docs/api.md
Co-authored-by: Bruce MacDonald <brucewmacdonald@gmail.com>

* Update api.md

---------
Signed-off-by: Matt Williams <m@technovangelist.com>
Co-authored-by: Bruce MacDonald <brucewmacdonald@gmail.com>
Co-authored-by: Michael Chiang <mchiang0610@users.noreply.github.com>

* linux gpu support
* handle multiple gpus
* add cuda docker image (#488)
---------
Co-authored-by: Michael Yang <mxyng@pm.me>

* remove c code
* pack llama.cpp
* use request context for llama_cpp
* let llama_cpp decide the number of threads to use
* stop llama runner when app stops
* remove sample count and duration metrics
* use go generate to get libraries
* tmp dir for running llm

The `stop` option to the generate API is a list of sequences that should cause generation to stop. Although these are commonly called "stop tokens", they do not necessarily correspond to LLM tokens (per the LLM's tokenizer). For example, if the caller sends a generate request with `"stop":["\n"]`, then generation should stop on any token containing `\n` (and trim `\n` from the output), not just if the token exactly matches `\n`. If `stop` were interpreted strictly as LLM tokens, then it would require callers of the generate API to know the LLM's tokenizer and enumerate many tokens in the `stop` list.

Fixes https://github.com/jmorganca/ollama/issues/295.

base_url value for Ollama object creation is corrected.

Signed-off-by: Matt Williams <m@technovangelist.com>

Co-authored-by: Bruce MacDonald <brucewmacdonald@gmail.com>

Co-authored-by: Bruce MacDonald <brucewmacdonald@gmail.com>

Co-authored-by: Bruce MacDonald <brucewmacdonald@gmail.com>

Signed-off-by: Matt Williams <m@technovangelist.com>

Co-authored-by: Michael Yang <mxyng@pm.me>

It is required to be adjusted for some models, see https://github.com/jmorganca/ollama/issues/320 for more context