Skip to content

GitLab

Unverified Commit 14e8bd88 authored by Yongtong Wu's avatar Yongtong Wu Committed by GitHub
Browse files

Free metadata_buffer_index after transfer finished (#5364)

parent adca585b
Hide whitespace changes
Inline Side-by-side
Showing with 5 additions and 0 deletions
python/sglang/srt/disaggregation/prefill.py
View file @ 14e8bd88
...@@ -226,6 +226,11 @@ class SchedulerDisaggregationPrefillMixin: ...@@ -226,6 +226,11 @@ class SchedulerDisaggregationPrefillMixin:
elif poll == KVPoll.Failed: elif poll == KVPoll.Failed:
raise Exception("Transferring failed") raise Exception("Transferring failed")
for req in done_reqs:
self.disagg_prefill_pending_queue.req_to_metadata_buffer_idx_allocator.free(
req.metadata_buffer_index
)
# Stream requests which have finished transfer # Stream requests which have finished transfer
self.stream_output(done_reqs, False, None) self.stream_output(done_reqs, False, None)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment