Unverified Commit 201c971e authored by Jialin Ouyang's avatar Jialin Ouyang Committed by GitHub
Browse files

[Perf][Easy] Early stop in request_block_hasher (#26112)


Signed-off-by: default avatarJialin Ouyang <Jialin.Ouyang@gmail.com>
parent e0986ea0
...@@ -585,6 +585,10 @@ def get_request_block_hasher( ...@@ -585,6 +585,10 @@ def get_request_block_hasher(
start_token_idx = len(request.block_hashes) * block_size start_token_idx = len(request.block_hashes) * block_size
num_tokens = request.num_tokens num_tokens = request.num_tokens
if start_token_idx + block_size > num_tokens:
# Early stop when there no new full blocks created.
return []
curr_mm_idx = 0 curr_mm_idx = 0
if start_token_idx > 0: if start_token_idx > 0:
# Set curr_mm_idx = -1 to indicate the last mm input. # Set curr_mm_idx = -1 to indicate the last mm input.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment