Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
63298ee1
Unverified
Commit
63298ee1
authored
Mar 07, 2026
by
Roy Huang
Committed by
GitHub
Mar 07, 2026
Browse files
[Bugfix][LMCache][KVConnector] fix potential memory leak in LMCache multiprocess mode (#35931)
parent
2dde535d
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
28 additions
and
0 deletions
+28
-0
vllm/distributed/kv_transfer/kv_connector/v1/lmcache_mp_connector.py
...buted/kv_transfer/kv_connector/v1/lmcache_mp_connector.py
+28
-0
No files found.
vllm/distributed/kv_transfer/kv_connector/v1/lmcache_mp_connector.py
View file @
63298ee1
...
...
@@ -721,6 +721,34 @@ class LMCacheMPConnector(KVConnectorBase_V1):
# Clean up lookup future in scheduler adapter
self
.
scheduler_adapter
.
cleanup_lookup_result
(
request
.
request_id
)
# Free locks on chunks that vLLM already computed and won't
# retrieve from LMCache.
if
tracker
.
num_lmcache_hit_blocks
>
0
:
if
not
condition
:
# No retrieve needed — free ALL locked chunks
free_end
=
tracker
.
num_lmcache_hit_blocks
*
self
.
vllm_block_size
else
:
# Note(Roy): Boundary misalignment between vLLM blocks and LMCache
# blocks is handled in free_lookup_locks. It makes sure that if
# the last vLLM computed block ends in the middle of a LMCache
# block, the end LMCache block is not freed (i.e., floor division)
# since it will still be needed by vLLM and such block's lock will
# be freed by vLLM's retrieve.
free_end
=
tracker
.
num_vllm_hit_blocks
*
self
.
vllm_block_size
if
free_end
>
0
:
self
.
scheduler_adapter
.
free_lookup_locks
(
token_ids
=
list
(
tracker
.
all_token_ids
),
start
=
0
,
end
=
free_end
,
request_id
=
request
.
request_id
,
)
logger
.
debug
(
"Free locks of tokens %d-%d since it is cached by vLLM."
,
0
,
free_end
,
)
def
build_connector_meta
(
self
,
scheduler_output
:
SchedulerOutput
)
->
KVConnectorMetadata
:
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment