Commit 074bac84 authored by Jesse Gross's avatar Jesse Gross Committed by Jesse Gross
Browse files

kvcache: Log batch size if we can't find a slot

In some cases, we can't find a cache slot when using sliding window
attention. It would be helpful in this (and other cases) to know what
the batch size is.

Bug #10127
parent 8e8f2c6d
...@@ -239,7 +239,7 @@ func (c *Causal) findStartLoc() (int, error) { ...@@ -239,7 +239,7 @@ func (c *Causal) findStartLoc() (int, error) {
} }
} }
return 0, fmt.Errorf("%w (length: %v)", ErrKvCacheFull, len(c.cells)) return 0, fmt.Errorf("%w (cache: %v batch: %v)", ErrKvCacheFull, len(c.cells), c.curBatchSize)
} }
func (c *Causal) updateSlidingWindow() { func (c *Causal) updateSlidingWindow() {
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment