GitLab

This adds new variants for arm64 specific to Jetson platforms

This should help speed things up a little

This adjusts linux to follow a similar model to windows with a discrete archive
(zip/tgz) to cary the primary executable, and dependent libraries. Runners are
still carried as payloads inside the main binary

Darwin retain the payload model where the go binary is fully self contained.

Ubuntu distros fail to install cuda drivers since aarch64 isn't valid

If we detect an NVIDIA GPU, but nvidia doesn't support the os/arch,
this will report a better error for the user and point them to docs
to self-install the drivers if possible.

Some users are experienging runner startup errors due
to not having these msvc redist libraries on their host

As of July 1st 2024: Could not resolve host: mirrorlist.centos.org
This is expected due to EOL dates.

We update the PATH on windows to get the CLI mapped, but this has
an unintended side effect of causing other apps that may use our bundled
DLLs to get terminated when we upgrade.

On an Ubuntu 24.04 computer with vmware installed, the sudo lshw command will get stuck. "Network interfaces" is always displayed

Ensure `nvidia` and `nvidia_uvm` kernel modules are loaded in `install.sh` script and at startup (#4652)

* ensure kernel modules are loaded in `install.sh` script and at startup

* indentation

* use `SUDO` variable

* restart if nouveau is detected

* consistent success message for AMD

The zip file encodes the OS and architecture, so keep the short exe name

This will make it simpler for CI to accumulate artifacts from prior steps

Now that the llm runner is an executable and not just a dll, more users are facing
problems with security policy configurations on windows that prevent users
writing to directories and then executing binaries from the same location.
This change removes payloads from the main executable on windows and shifts them
over to be packaged in the installer and discovered based on the executables location.
This also adds a new zip file for people who want to "roll their own" installation model.

This should hopefully only be a temporary workaround until Rocky 8
picks up GCC 10.4 which fixes the NVCC bug

This uplevels the integration tests to run the server which can allow
testing an existing server, or a remote server.

Flesh out our github actions CI so we can build official releaes.

This refines where we extract the LLM libraries to by adding a new
OLLAMA_HOME env var, that defaults to `~/.ollama` The logic was already
idempotenent, so this should speed up startups after the first time a
new release is deployed.  It also cleans up after itself.

We now build only a single ROCm version (latest major) on both windows
and linux.  Given the large size of ROCms tensor files, we split the
dependency out.  It's bundled into the installer on windows, and a
separate download on windows.  The linux install script is now smart and
detects the presence of AMD GPUs and looks to see if rocm v6 is already
present, and if not, then downloads our dependency tar file.

For Linux discovery, we now use sysfs and check each GPU against what
ROCm supports so we can degrade to CPU gracefully instead of having
llama.cpp+rocm assert/crash on us.  For Windows, we now use go's windows
dynamic library loading logic to access the amdhip64.dll APIs to query
the GPU information.

On OpenSUSE, ollama needs to be a member of the video group
to access the GPU

Allow overriding the platform, image name, and tag latest for
standard and rocm images.

* remove `-w -s` linker flags on windows

* use `zip` for windows installer compression

Also fixes a few fit-and-finish items for better developer experience

This will be useful for our automated test riggig, and may be useful for
advanced users who want to "roll their own" system service