[Perf] Improved perf for resolve_chat_template_content_format (#20065)

Signed-off-by: Ilya Lavrenov <ilya.lavrenov@cerebras.net>

[Perf] Improved perf for resolve_chat_template_content_format (#20065)
Signed-off-by: Ilya Lavrenov <ilya.lavrenov@cerebras.net>
aa0dc77e · Ilya Lavrenov · GitHub · 4ab3ac28 · aa0dc77e
Unverified Commit aa0dc77e authored Jun 27, 2025 by Ilya Lavrenov Committed by GitHub Jun 27, 2025
Hide whitespace changes
Inline Side-by-side

Showing with 1 addition and 0 deletions

vllm/entrypoints/chat_utils.py vllm/entrypoints/chat_utils.py +1 -0

No files found.
--- a/vllm/entrypoints/chat_utils.py
+++ b/vllm/entrypoints/chat_utils.py
@@ -293,6 +293,7 @@ def _try_extract_ast(chat_template: str) -> Optional[jinja2.nodes.Template]:
        return None


+@lru_cache(maxsize=32)
 def _detect_content_format(
    chat_template: str,
    *,