Skip to content

GitLab

Unverified Commit 358ae356 authored by Shangming Cai's avatar Shangming Cai Committed by GitHub
Browse files

Tiny fix eos handling for PD disaggregation (#12334) 


Signed-off-by: default avatarShangming Cai <csmthu@gmail.com>
parent 41c10e67
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
python/sglang/srt/disaggregation/decode.py
View file @ 358ae356
......@@ -776,7 +776,7 @@ class DecodeTransferQueue:
# special handling for corner cases
should_finish = (
decode_req.req.sampling_params.max_new_tokens == 1
or output_id in decode_req.req.eos_token_ids
or decode_req.req.output_ids[-1] in decode_req.req.eos_token_ids
)
if should_finish:
# finish immediately
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment