GitLab

This reverts commit 755b4e4f.

pointer deref's weren't correct on a few libraries, which explains
some crashes on older systems or miswired symlinks for discovery libraries.

This reverts commit 163cd3e7.

We update the PATH on windows to get the CLI mapped, but this has
an unintended side effect of causing other apps that may use our bundled
DLLs to get terminated when we upgrade.

Signed-off-by: Lei Jitang <leijitang@outlook.com>

* gpu: add env var for detecting intel oneapi gpus

* fix build error

Also removes an unused overall count variable

Signed-off-by: Lei Jitang <leijitang@outlook.com>

This should aid in troubleshooting by capturing and reporting the GPU
settings at startup in the logs along with all the other server settings.

Implement support for GPU env var workarounds, and leverage
this for the Vega RX 56 which needs
HSA_ENABLE_SDMA=0 set to work properly

This library will give us the most reliable free VRAM reporting on windows
to enable concurrent model scheduling.

Still not complete, needs some refinement to our prediction to understand the
discrete GPUs available space so we can see how many layers fit in each one
since we can't split one layer across multiple GPUs we can't treat free space
as one logical block

Now that we call the GPU discovery routines many times to
update memory, this splits initial discovery from free memory
updating.

The amdgpu drivers free VRAM reporting omits some other apps, so leverage the
upstream DRM driver which keeps better tabs on things

This reverts commit 476fb8e8.

* fix oneapi errors on windows 10

Under stress scenarios we're seeing OOMs so this should help stabilize
the allocations under heavy concurrency stress.

The GPU drivers take a while to update their free memory reporting, so we need
to wait until the values converge with what we're expecting before proceeding
to start another runner in order to get an accurate picture.

This cleans up the logging for GPU discovery a bit, and can
serve as a foundation to report GPU information in a future UX.

Trying to live off the land for cuda libraries was not the right strategy.  We need to use the version we compiled against to ensure things work properly

This moves all the env var reading into one central module
and logs the loaded config once at startup which should
help in troubleshooting user server logs

For some reason this library gives incorrect GPU information, so skip it

We're seeing some corner cases with cudart which might be resolved by
switching to the driver API which comes bundled with the driver package