Skip to content

GitLab

Unverified Commit 358ae356 authored by Shangming Cai's avatar Shangming Cai Committed by GitHub
Browse files

Tiny fix eos handling for PD disaggregation (#12334) 


Signed-off-by: default avatarShangming Cai <csmthu@gmail.com>
parent 41c10e67
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
python/sglang/srt/disaggregation/decode.py
View file @ 358ae356
...@@ -776,7 +776,7 @@ class DecodeTransferQueue: ...@@ -776,7 +776,7 @@ class DecodeTransferQueue:
# special handling for corner cases # special handling for corner cases
should_finish = ( should_finish = (
decode_req.req.sampling_params.max_new_tokens == 1 decode_req.req.sampling_params.max_new_tokens == 1
or output_id in decode_req.req.eos_token_ids or decode_req.req.output_ids[-1] in decode_req.req.eos_token_ids
) )
if should_finish: if should_finish:
# finish immediately # finish immediately
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment