Unverified Commit 8c8779cd authored by yukavio's avatar yukavio Committed by GitHub
Browse files

[Fix] fix retract error in eagle speculative decoding (#2711)


Co-authored-by: default avatarkavioyu <kavioyu@tencent.com>
parent 1775b963
......@@ -601,11 +601,11 @@ class EagleVerifyInput(SpecInfo):
req.check_finished()
if req.finished():
draft_input.has_finished = True
finished_extend_len[req.rid] = verified_len + 1
else:
new_accept_index.append(accept_index[low : low + verified_len + 1])
unfinished_index.append(i)
low += verified_len + 1
finished_extend_len[req.rid] = verified_len + 1
if len(new_accept_index) > 0:
new_accept_index = torch.cat(new_accept_index, dim=0)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment