"examples/pytorch/vscode:/vscode.git/clone" did not exist on "9fee20b91dba3804a7a19f327b2b1d2407d93874"
Unverified Commit 9b5f0f64 authored by Liangsheng Yin's avatar Liangsheng Yin Committed by GitHub
Browse files

Fix tiny misalign with previous truncation setting in tokenizer_manager (#9430)

parent 70bb066e
...@@ -566,7 +566,7 @@ class TokenizerManager: ...@@ -566,7 +566,7 @@ class TokenizerManager:
) -> None: ) -> None:
"""Validates that the input token count and the requested token count doesn't exceed the model's context length.""" """Validates that the input token count and the requested token count doesn't exceed the model's context length."""
# FIXME: unify the length validation logic with the one in the scheduler. # FIXME: unify the length validation logic with the one in the scheduler.
_max_req_len = self.context_len - 1 _max_req_len = self.context_len
input_token_num = len(input_ids) if input_ids is not None else 0 input_token_num = len(input_ids) if input_ids is not None else 0
if input_token_num >= self.context_len: if input_token_num >= self.context_len:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment