Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
57c84ff1
Unverified
Commit
57c84ff1
authored
Mar 06, 2026
by
cong-or
Committed by
GitHub
Mar 05, 2026
Browse files
perf: add __slots__ to KVCacheBlock (#36164)
Signed-off-by:
cong-or
<
conchubhar.gannon@gmail.com
>
parent
e68de8ad
Changes
2
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
13 additions
and
1 deletion
+13
-1
tests/v1/core/test_kv_cache_utils.py
tests/v1/core/test_kv_cache_utils.py
+12
-0
vllm/v1/core/kv_cache_utils.py
vllm/v1/core/kv_cache_utils.py
+1
-1
No files found.
tests/v1/core/test_kv_cache_utils.py
View file @
57c84ff1
...
...
@@ -202,6 +202,18 @@ def test_kv_cache_block():
assert
block
.
block_hash
is
None
def
test_kv_cache_block_uses_slots
():
block
=
KVCacheBlock
(
block_id
=
0
)
# Slots eliminate per-instance __dict__, saving ~264 bytes per block.
# At 100K+ blocks this avoids tens of MB of overhead and GC pressure.
assert
not
hasattr
(
block
,
"__dict__"
)
# Verify that slots actually prevent dynamic attribute assignment.
with
pytest
.
raises
(
AttributeError
):
block
.
unexpected_field
=
True
def
test_free_kv_cache_block_queue_initialization
():
# Test with a single block
block
=
KVCacheBlock
(
block_id
=
0
)
...
...
vllm/v1/core/kv_cache_utils.py
View file @
57c84ff1
...
...
@@ -106,7 +106,7 @@ def init_none_hash(hash_fn: Callable[[Any], bytes]):
NONE_HASH
=
BlockHash
(
hash_fn
(
hash_seed
))
@
dataclass
@
dataclass
(
slots
=
True
)
class
KVCacheBlock
:
"""KV-cache block metadata."""
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment