vllm/v1/engine/detokenizer.py · 2c301ee2eb2d60015936c5e34c80fa62d3c2d37d · OpenDAS / vllm_cscc · GitLab

Find file Blame History Permalink

[Bugfix] Fix Incremental Detokenization with `tokenizers == 0.22.0` (#24159) · 2c301ee2

Fanli Lin authored Sep 04, 2025

Signed-off-by: Fanli Lin <fanli.lin@intel.com>
Signed-off-by: Fanli Lin <fanli0116@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

2c301ee2

detokenizer.py 11.1 KB