GitLab

Trying to live off the land for cuda libraries was not the right strategy.  We need to use the version we compiled against to ensure things work properly

Fix stale test logic

The model processing was recently changed to be deferred but
this test scenario hadn't been adjusted for that change in behavior.

Skip PhysX cudart library

Add CUDA Driver API for GPU discovery

Co-authored-by: Bruce MacDonald <brucewmacdonald@gmail.com>

* fix llava models not working after first request

* individual requests only for llava models

Centralize server config handling

This moves all the env var reading into one central module
and logs the loaded config once at startup which should
help in troubleshooting user server logs

---------
Co-authored-by: Patrick Devine <patrick@infrahq.com>

Make maximum pending request configurable

This also bumps up the default to be 50 queued requests
instead of 10.

Explain the 2 different windows download options

omit prompt and generate settings from final response

Fix lint warnings

rename parser to model/file

* Update 'llama2' -> 'llama3' in most places

---------
Co-authored-by: Patrick Devine <patrick@infrahq.com>

For some reason this library gives incorrect GPU information, so skip it

Soften timeouts on sched unit tests

This gives us more headroom on the scheduler tests to tamp
down some flakes.

refactor modelfile parser

fix line ending

replace CRLF with LF

* Add a /clear command

* change help messages

---------
Co-authored-by: Patrick Devine <patrick@infrahq.com>