Skip to content

GitLab

Unverified Commit 2f192835 authored by youkaichao's avatar youkaichao Committed by GitHub
Browse files

[Core] latency optimization (#3890)

parent 95baec82
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
vllm/core/block_manager_v1.py
View file @ 2f192835
...@@ -328,7 +328,7 @@ class BlockSpaceManagerV1(BlockSpaceManager): ...@@ -328,7 +328,7 @@ class BlockSpaceManagerV1(BlockSpaceManager):
self, self,
seq: Sequence, seq: Sequence,
) -> bool: ) -> bool:
token_ids_len = len(seq.data.get_token_ids()) token_ids_len = seq.data.get_len()
return token_ids_len > 0 and token_ids_len % seq.block_size == 0 return token_ids_len > 0 and token_ids_len % seq.block_size == 0
def _maybe_promote_last_block( def _maybe_promote_last_block(
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment