Skip to content

GitLab

Switch branch/tag
  1. 28 Apr, 2025 1 commit
  3. 22 Apr, 2025 1 commit
    • Devon Rifkin's avatar
      increase default context length to 4096 (#10364) · 424f6486
      Devon Rifkin authored 
      * increase default context length to 4096

We lower the default numParallel from 4 to 2 and use these "savings" to
double the default context length from 2048 to 4096.

We're memory neutral in cases when we previously would've used
numParallel == 4, but we add the following mitigation to handle some
cases where we would have previously fallen back to 1x2048 due to low
VRAM: we decide between 2048 and 4096 using a runtime check, choosing
2048 if we're on a one GPU system with total VRAM of <= 4 GB. We
purposefully don't check the available VRAM because we don't want the
context window size to change unexpectedly based on the available VRAM.

We plan on making the default even larger, but this is a relatively
low-risk change we can make to quickly double it.

* fix tests

add an explicit context length so they don't get truncated. The code
that converts -1 from being a signal for doing a runtime check isn't
running as part of these tests.

* tweak small gpu message

* clarify context length default

also make it actually show up in `ollama serve --help`
      424f6486
  5. 27 Mar, 2025 1 commit
  7. 13 Mar, 2025 1 commit
  9. 10 Mar, 2025 1 commit
  11. 07 Feb, 2025 2 commits
  13. 03 Dec, 2024 1 commit
  15. 25 Sep, 2024 1 commit
  17. 18 Sep, 2024 1 commit
  19. 10 Sep, 2024 1 commit
  21. 02 Sep, 2024 1 commit
  23. 23 Aug, 2024 1 commit
  25. 29 Jul, 2024 1 commit
  27. 23 Jul, 2024 1 commit
  29. 10 Jul, 2024 1 commit
    • Daniel Hiltgen's avatar
      Bump ROCm on windows to 6.1.2 · 1f50356e
      Daniel Hiltgen authored 
      This also adjusts our algorithm to favor our bundled ROCm.
I've confirmed VRAM reporting still doesn't work properly so we
can't yet enable concurrency by default.
      1f50356e
  31. 02 Jul, 2024 1 commit
  33. 28 Jun, 2024 1 commit
  35. 21 May, 2024 1 commit
  37. 14 May, 2024 1 commit
  39. 06 May, 2024 1 commit
  41. 05 May, 2024 1 commit
  43. 03 May, 2024 1 commit
  45. 24 Apr, 2024 1 commit
  47. 26 Mar, 2024 1 commit
  49. 21 Mar, 2024 2 commits
  51. 20 Mar, 2024 1 commit
  53. 18 Mar, 2024 2 commits
  55. 12 Mar, 2024 1 commit
  57. 21 Feb, 2024 1 commit
  59. 20 Feb, 2024 2 commits
  61. 19 Feb, 2024 2 commits
  63. 16 Feb, 2024 1 commit
  65. 22 Jan, 2024 1 commit
  67. 22 Dec, 2023 2 commits