Fix potential eos bug on decode instance when PD is enabled (#12206)

Signed-off-by: Shangming Cai <csmthu@gmail.com>

Fix potential eos bug on decode instance when PD is enabled (#12206)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
41a11335 · Shangming Cai · GitHub · a1f2dc90 · 41a11335
Unverified Commit 41a11335 authored Oct 28, 2025 by Shangming Cai Committed by GitHub Oct 28, 2025
Hide whitespace changes
Inline Side-by-side

Showing with 6 additions and 2 deletions

python/sglang/srt/disaggregation/decode.py python/sglang/srt/disaggregation/decode.py +6 -2

No files found.
--- a/python/sglang/srt/disaggregation/decode.py
+++ b/python/sglang/srt/disaggregation/decode.py
@@ -773,8 +773,12 @@ class DecodeTransferQueue:
                indices_to_remove.add(i)
                decode_req.req.time_stats.wait_queue_entry_time = time.perf_counter()
-                # special handling for sampling_params.max_new_tokens == 1
+                # special handling for corner cases
-                if decode_req.req.sampling_params.max_new_tokens == 1:
+                should_finish = (
+                    decode_req.req.sampling_params.max_new_tokens == 1
+                    or output_id in decode_req.req.eos_token_ids
+                )
+                if should_finish:
                    # finish immediately
                    decode_req.req.time_stats.forward_entry_time = (
                        decode_req.req.time_stats.completion_time