Unverified Commit 17536e7e authored by Byron Hsu's avatar Byron Hsu Committed by GitHub
Browse files

Fix edge case for truncated (#1747)

parent 1f26e8b8
...@@ -416,7 +416,7 @@ class Scheduler: ...@@ -416,7 +416,7 @@ class Scheduler:
) )
# Truncate prompts that are too long # Truncate prompts that are too long
if len(req.origin_input_ids) >= self.max_req_input_len: if len(req.origin_input_ids) > self.max_req_input_len:
logger.warning( logger.warning(
"Request length is longer than the KV cache pool size or " "Request length is longer than the KV cache pool size or "
"the max context length. Truncated!!!" "the max context length. Truncated!!!"
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment