1. 11 Jan, 2024 1 commit
    • Daniel Hiltgen's avatar
      Support multiple variants for a given llm lib type · 8da7bef0
      Daniel Hiltgen authored
      In some cases we may want multiple variants for a given GPU type or CPU.
      This adds logic to have an optional Variant which we can use to select
      an optimal library, but also allows us to try multiple variants in case
      some fail to load.
      
      This can be useful for scenarios such as ROCm v5 vs v6 incompatibility
      or potentially CPU features.
      8da7bef0
  2. 09 Jan, 2024 1 commit
  3. 04 Jan, 2024 1 commit
    • Daniel Hiltgen's avatar
      Load dynamic cpu lib on windows · e9ce91e9
      Daniel Hiltgen authored
      On linux, we link the CPU library in to the Go app and fall back to it
      when no GPU match is found. On windows we do not link in the CPU library
      so that we can better control our dependencies for the CLI.  This fixes
      the logic so we correctly fallback to the dynamic CPU library
      on windows.
      e9ce91e9
  4. 02 Jan, 2024 1 commit
    • Daniel Hiltgen's avatar
      Switch windows build to fully dynamic · d966b730
      Daniel Hiltgen authored
      Refactor where we store build outputs, and support a fully dynamic loading
      model on windows so the base executable has no special dependencies thus
      doesn't require a special PATH.
      d966b730