Commit 3d9498a4 authored by Jesse Gross's avatar Jesse Gross Committed by Jesse Gross
Browse files

ollamarunner: Use correct constant to remove cache entries

The correct constant to remove all entries to the end of the sequence
for the Ollama engine is math.MaxInt32. -1 is used by the old engine.

The impact of this is currently minimal because it would only occur
in situations that are not supported by the implemented models or
rarely used options.
parent 3098c8b2
...@@ -284,7 +284,7 @@ func (c *InputCache) ShiftCacheSlot(slot *InputCacheSlot, numKeep int32) error { ...@@ -284,7 +284,7 @@ func (c *InputCache) ShiftCacheSlot(slot *InputCacheSlot, numKeep int32) error {
copy(newInputs[numKeep:], slot.Inputs[numKeep+discard:]) copy(newInputs[numKeep:], slot.Inputs[numKeep+discard:])
// Reset the cache // Reset the cache
_ = c.cache.Remove(slot.Id, 0, -1) _ = c.cache.Remove(slot.Id, 0, math.MaxInt32)
slot.Inputs = []input.Input{} slot.Inputs = []input.Input{}
// Return error with inputs that need to be reprocessed // Return error with inputs that need to be reprocessed
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment