Skip to content

GitLab

Unverified Commit 97f92c6b authored by Nicolò Lucchesi's avatar Nicolò Lucchesi Committed by GitHub
Browse files

[KVConnector] Skip `register_kv_caches` on profiling (#38558) 


Signed-off-by: default avatarNickLucche <nlucches@redhat.com>
parent 46f02e00
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
vllm/v1/worker/gpu_model_runner.py
View file @ 97f92c6b
...@@ -6844,7 +6844,7 @@ class GPUModelRunner( ...@@ -6844,7 +6844,7 @@ class GPUModelRunner(
# group # group
self.drafter.validate_same_kv_cache_group(kv_cache_config) self.drafter.validate_same_kv_cache_group(kv_cache_config)
if has_kv_transfer_group(): if has_kv_transfer_group() and not is_profiling:
kv_transfer_group = get_kv_transfer_group() kv_transfer_group = get_kv_transfer_group()
if self.cross_layers_kv_cache is not None: if self.cross_layers_kv_cache is not None:
assert self.cross_layers_attn_backend is not None assert self.cross_layers_attn_backend is not None
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment