Unverified Commit 2f385183 authored by Kathy Yu's avatar Kathy Yu Committed by GitHub
Browse files

[Bugfix] Free cross attention block table for preempted-for-recompute sequence group. (#10013)


Signed-off-by: default avatarKathy Yu <feiyangyu@google.com>
parent 84c35c37
......@@ -1579,6 +1579,7 @@ class Scheduler:
seq.status = SequenceStatus.WAITING
self.free_seq(seq)
seq.reset_state_for_recompute()
self._free_seq_group_cross_attn_blocks(seq_group)
def _preempt_by_swap(
self,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment