tests/tokenization/test_mistral_tokenizer.py · b2c1d294faca96643dbc2413d604ca160f458f0d · OpenDAS / vllm_cscc · GitLab

Find file Blame History Permalink

[BUGFIX] MistralTokenizer._call__ adds an invalid EOS token (#29607) · b2c1d294

Julien Denize authored Nov 28, 2025

Signed-off-by: Julien Denize <julien.denize@mistral.ai>
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

b2c1d294

test_mistral_tokenizer.py 70 KB