Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
bfe12894
Commit
bfe12894
authored
Sep 14, 2025
by
xuxzh1
🎱
Browse files
完善释放包括cpu和tensor_id释放
parent
0627b53a
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
12 additions
and
3 deletions
+12
-3
vllm/distributed/kv_transfer/kv_connector/v1/p2p/p2p_nccl_connector.py
...ted/kv_transfer/kv_connector/v1/p2p/p2p_nccl_connector.py
+12
-3
No files found.
vllm/distributed/kv_transfer/kv_connector/v1/p2p/p2p_nccl_connector.py
View file @
bfe12894
...
@@ -215,9 +215,18 @@ class P2pNcclConnector(KVConnectorBase_V1):
...
@@ -215,9 +215,18 @@ class P2pNcclConnector(KVConnectorBase_V1):
inject_kv_into_layer
(
kv_cache_layer
,
kv_cache
,
inject_kv_into_layer
(
kv_cache_layer
,
kv_cache
,
request
.
slot_mapping
,
request
.
request_id
)
request
.
slot_mapping
,
request
.
request_id
)
tensor
=
self
.
p2p_nccl_engine
.
recv_store
.
pop
(
request
.
request_id
+
"#"
+
layer_name
,
None
)
tensor_id
=
request
.
request_id
+
"#"
+
layer_name
if
tensor
is
not
None
:
if
tensor_id
in
self
.
p2p_nccl_engine
.
recv_store
:
del
tensor
tensor
=
self
.
p2p_nccl_engine
.
recv_store
.
pop
(
tensor_id
,
None
)
self
.
p2p_nccl_engine
.
send_request_id_to_tensor_ids
.
pop
(
request
.
request_id
,
None
)
self
.
p2p_nccl_engine
.
recv_request_id_to_tensor_ids
.
pop
(
request
.
request_id
,
None
)
addr
=
0
if
isinstance
(
tensor
,
tuple
):
addr
,
_
,
_
=
tensor
self
.
p2p_nccl_engine
.
pool
.
free
(
addr
)
def
wait_for_layer_load
(
self
,
layer_name
:
str
)
->
None
:
def
wait_for_layer_load
(
self
,
layer_name
:
str
)
->
None
:
"""Blocking until the KV for a specific layer is loaded into vLLM's
"""Blocking until the KV for a specific layer is loaded into vLLM's
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment