Skip to content

GitLab

Unverified Commit bb6d8c21 authored by Nick Hill's avatar Nick Hill Committed by GitHub
Browse files

[Bugfix] Catch and log invalid token ids in detokenizer #2 (#26445) 


Signed-off-by: default avatarNick Hill <nhill@redhat.com>
parent ebf6ef1a
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
vllm/v1/engine/detokenizer.py
View file @ bb6d8c21
...@@ -227,10 +227,10 @@ class FastIncrementalDetokenizer(BaseIncrementalDetokenizer): ...@@ -227,10 +227,10 @@ class FastIncrementalDetokenizer(BaseIncrementalDetokenizer):
def _protected_step(self, next_token_id: int) -> Optional[str]: def _protected_step(self, next_token_id: int) -> Optional[str]:
try: try:
token = self.stream.step(self.tokenizer, next_token_id) token = self.stream.step(self.tokenizer, next_token_id)
except OverflowError: except (OverflowError, TypeError):
# Handle rare observed overflow, still to be diagnosed. # Handle rare observed overflow, still to be diagnosed.
# See https://github.com/vllm-project/vllm/issues/21951. # See https://github.com/vllm-project/vllm/issues/21951.
logger.exception("Encountered invalid token id: %d", next_token_id) logger.exception("Encountered invalid token id: %r", next_token_id)
token = None token = None
except Exception as e: except Exception as e:
if not str(e).startswith(INVALID_PREFIX_ERR_MSG): if not str(e).startswith(INVALID_PREFIX_ERR_MSG):
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment