"docs/vscode:/vscode.git/clone" did not exist on "24d6ea8afdb13ceee95b36645ba61a641f9a2f7f"
Unverified Commit 2f385183 authored by Kathy Yu's avatar Kathy Yu Committed by GitHub
Browse files

[Bugfix] Free cross attention block table for preempted-for-recompute sequence group. (#10013)


Signed-off-by: default avatarKathy Yu <feiyangyu@google.com>
parent 84c35c37
...@@ -1579,6 +1579,7 @@ class Scheduler: ...@@ -1579,6 +1579,7 @@ class Scheduler:
seq.status = SequenceStatus.WAITING seq.status = SequenceStatus.WAITING
self.free_seq(seq) self.free_seq(seq)
seq.reset_state_for_recompute() seq.reset_state_for_recompute()
self._free_seq_group_cross_attn_blocks(seq_group)
def _preempt_by_swap( def _preempt_by_swap(
self, self,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment