Skip to content

GitLab

Unverified Commit e3f3aee6 authored by Nicolò Lucchesi's avatar Nicolò Lucchesi Committed by GitHub
Browse files

[Misc] Avoid cuda graph log when sizes still match (#18202) 


Signed-off-by: default avatarNickLucche <nlucches@redhat.com>
parent 92540529
Show whitespace changes
Inline Side-by-side
Showing with 6 additions and 5 deletions
vllm/config.py
View file @ e3f3aee6
......@@ -3950,11 +3950,12 @@ class CompilationConfig:
self.cudagraph_capture_sizes = cudagraph_capture_sizes
else:
# de-duplicate the sizes provided by the config
self.cudagraph_capture_sizes = list(
set(self.cudagraph_capture_sizes))
dedup_sizes = list(set(self.cudagraph_capture_sizes))
if len(dedup_sizes) < len(self.cudagraph_capture_sizes):
logger.info(("cudagraph sizes specified by model runner"
" %s is overridden by config %s"),
cudagraph_capture_sizes, self.cudagraph_capture_sizes)
cudagraph_capture_sizes, dedup_sizes)
self.cudagraph_capture_sizes = dedup_sizes
computed_compile_sizes = []
if self.compile_sizes is not None:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment