Skip to content

GitLab

"apps/vscode:/vscode.git/clone" did not exist on "1b9bc16b1a57c6e7957c3fd74d89d1b206fc5bb8"
Switch branch/tag
  1. 07 Oct, 2025 1 commit
    • Daniel Hiltgen's avatar
      Bring back escape valve for llm libraries and fix Jetpack6 crash (#12529) · bd15eba4
      Daniel Hiltgen authored 
      * Bring back escape valve for llm libraries

If the new discovery logic picks the wrong library, this gives users the
ability to force a specific one using the same pattern as before. This
can also potentially speed up bootstrap discovery if one of the libraries
takes a long time to load and ultimately bind to no devices.  For example
unsupported AMD iGPUS can sometimes take a while to discover and rule out.

* Bypass extra discovery on jetpack systems

On at least Jetpack6, cuda_v12 appears to expose the iGPU, but crashes later on in
cublasInit so if we detect a Jetpack, short-circuit and use that variant.
      bd15eba4
  3. 08 Sep, 2025 1 commit
  5. 28 Jul, 2025 1 commit
  7. 23 Jun, 2025 1 commit
    • Daniel Hiltgen's avatar
      Re-remove cuda v11 (#10694) · 1c6669e6
      Daniel Hiltgen authored 
      * Re-remove cuda v11

Revert the revert - drop v11 support requiring drivers newer than Feb 23

This reverts commit c6bcdc42.

* Simplify layout

With only one version of the GPU libraries, we can simplify things down somewhat.  (Jetsons still require special handling)

* distinct sbsa variant for linux arm64

This avoids accidentally trying to load the sbsa cuda libraries on
a jetson system which results in crashes.

* temporary prevent rocm+cuda mixed loading
      1c6669e6
  9. 13 May, 2025 1 commit
  11. 07 May, 2025 1 commit
    • Daniel Hiltgen's avatar
      remove cuda v11 (#10569) · fa393554
      Daniel Hiltgen authored 
      This reduces the size of our Windows installer payloads by ~256M by dropping
support for nvidia drivers older than Feb 2023.  Hardware support is unchanged.

Linux default bundle sizes are reduced by ~600M to 1G.
      fa393554
  13. 08 Apr, 2025 1 commit
  15. 25 Mar, 2025 1 commit
  17. 25 Feb, 2025 1 commit
  19. 07 Feb, 2025 1 commit
  21. 10 Dec, 2024 1 commit
  23. 21 Nov, 2024 1 commit
  25. 12 Nov, 2024 2 commits
  27. 11 Sep, 2024 1 commit
    • Daniel Hiltgen's avatar
      Verify permissions for AMD GPU (#6736) · 9246e6dd
      Daniel Hiltgen authored 
      This adds back a check which was lost many releases back to verify /dev/kfd permissions
which when lacking, can lead to confusing failure modes of:
  "rocBLAS error: Could not initialize Tensile host: No devices found"

This implementation does not hard fail the serve command but instead will fall back to CPU
with an error log.  In the future we can include this in the GPU discovery UX to show
detected but unsupported devices we discovered.
      9246e6dd
  29. 05 Aug, 2024 1 commit
    • frob's avatar
      Disable paging for journalctl (#6154) · b73b0940
      frob authored 
      Users using `journalctl` to get logs for issue logging sometimes don't realize that paging is causing information to be missed.
      b73b0940
  31. 04 Jul, 2024 1 commit
  33. 03 Jul, 2024 1 commit
    • Daniel Hiltgen's avatar
      Better nvidia GPU discovery logging · ef757da2
      Daniel Hiltgen authored 
      Refine the way we log GPU discovery to improve the non-debug
output, and report more actionable log messages when possible
to help users troubleshoot on their own.
      ef757da2
  35. 19 Jun, 2024 1 commit
  37. 23 May, 2024 1 commit
  39. 21 May, 2024 1 commit
  41. 20 May, 2024 1 commit
  43. 09 May, 2024 1 commit
  45. 01 Apr, 2024 1 commit
    • Daniel Hiltgen's avatar
      Safeguard for noexec · 0a74cb31
      Daniel Hiltgen authored 
      We may have users that run into problems with our current
payload model, so this gives us an escape valve.
      0a74cb31
  47. 21 Mar, 2024 1 commit
  49. 15 Mar, 2024 1 commit
  51. 11 Mar, 2024 1 commit
  53. 07 Mar, 2024 2 commits
    • Daniel Hiltgen's avatar
      Refined ROCm troubleshooting docs · 69f02278
      Daniel Hiltgen authored
      69f02278
    • Daniel Hiltgen's avatar
      Revamp ROCm support · 6c5ccb11
      Daniel Hiltgen authored 
      This refines where we extract the LLM libraries to by adding a new
OLLAMA_HOME env var, that defaults to `~/.ollama` The logic was already
idempotenent, so this should speed up startups after the first time a
new release is deployed.  It also cleans up after itself.

We now build only a single ROCm version (latest major) on both windows
and linux.  Given the large size of ROCms tensor files, we split the
dependency out.  It's bundled into the installer on windows, and a
separate download on windows.  The linux install script is now smart and
detects the presence of AMD GPUs and looks to see if rocm v6 is already
present, and if not, then downloads our dependency tar file.

For Linux discovery, we now use sysfs and check each GPU against what
ROCm supports so we can degrade to CPU gracefully instead of having
llama.cpp+rocm assert/crash on us.  For Windows, we now use go's windows
dynamic library loading logic to access the amdhip64.dll APIs to query
the GPU information.
      6c5ccb11
  55. 15 Feb, 2024 1 commit
  57. 29 Jan, 2024 1 commit
  59. 11 Jan, 2024 1 commit
    • Daniel Hiltgen's avatar
      Build multiple CPU variants and pick the best · d88c527b
      Daniel Hiltgen authored 
      This reduces the built-in linux version to not use any vector extensions
which enables the resulting builds to run under Rosetta on MacOS in
Docker.  Then at runtime it checks for the actual CPU vector
extensions and loads the best CPU library available
      d88c527b
  61. 22 Dec, 2023 1 commit