Skip to content

GitLab

Unverified Commit fb71725c authored by Ying Sheng's avatar Ying Sheng Committed by GitHub
Browse files

Fix a bug in schedule_policy (#6276)

parent 912788c0
Show whitespace changes
Inline Side-by-side
Showing with 3 additions and 0 deletions
python/sglang/srt/managers/schedule_policy.py
View file @ fb71725c
...@@ -468,6 +468,9 @@ class PrefillAdder: ...@@ -468,6 +468,9 @@ class PrefillAdder:
return AddReqResult.OTHER return AddReqResult.OTHER
with self._lock_node(req.last_node): with self._lock_node(req.last_node):
if total_tokens > self.rem_total_tokens:
return AddReqResult.NO_TOKEN
if ( if (
enable_hierarchical_cache enable_hierarchical_cache
and req.last_node_global is not None and req.last_node_global is not None
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment