GitLab

- FROM /path/to/{safetensors,pytorch}
- FROM /path/to/fp{16,32}.bin
- FROM model:fp{16,32}

Add -e to viewing logs in order to show end of ollama logs

User our bundled libraries (cuda) instead of the host library

* Update api.md

Changed the calculation of tps (token/s) in the documentation

* Update docs/api.md

---------
Co-authored-by: Jeffrey Morgan <jmorganca@gmail.com>

Support Fedoras standard ROCm location

Trying to live off the land for cuda libraries was not the right strategy.  We need to use the version we compiled against to ensure things work properly

Fix stale test logic

The model processing was recently changed to be deferred but
this test scenario hadn't been adjusted for that change in behavior.

Skip PhysX cudart library

Add CUDA Driver API for GPU discovery

Co-authored-by: Bruce MacDonald <brucewmacdonald@gmail.com>

* fix llava models not working after first request

* individual requests only for llava models

Centralize server config handling

This moves all the env var reading into one central module
and logs the loaded config once at startup which should
help in troubleshooting user server logs

---------
Co-authored-by: Patrick Devine <patrick@infrahq.com>

Make maximum pending request configurable

This also bumps up the default to be 50 queued requests
instead of 10.

Explain the 2 different windows download options

omit prompt and generate settings from final response

Fix lint warnings

rename parser to model/file

* Update 'llama2' -> 'llama3' in most places

---------
Co-authored-by: Patrick Devine <patrick@infrahq.com>

For some reason this library gives incorrect GPU information, so skip it