Skip to content

GitLab

Unverified Commit 739e03b3 authored by Pete Savage's avatar Pete Savage Committed by GitHub
Browse files

[Bugfix] Fixed mistral tokenizer path when pointing to file (#17457) 


Signed-off-by: default avatarPete Savage <psavage@redhat.com>
parent da4e7687
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
vllm/transformers_utils/tokenizers/mistral.py
View file @ 739e03b3
...@@ -227,6 +227,7 @@ class MistralTokenizer(TokenizerBase): ...@@ -227,6 +227,7 @@ class MistralTokenizer(TokenizerBase):
else: else:
assert Path( assert Path(
path_or_repo_id).is_file(), f"Invalid path: {path_or_repo_id}" path_or_repo_id).is_file(), f"Invalid path: {path_or_repo_id}"
tokenizer_file = str(Path(path_or_repo_id))
from mistral_common.tokens.tokenizers.mistral import ( from mistral_common.tokens.tokenizers.mistral import (
MistralTokenizer as PublicMistralTokenizer) MistralTokenizer as PublicMistralTokenizer)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment