Unverified Commit a332b845 authored by Harry Mellor's avatar Harry Mellor Committed by GitHub
Browse files

[CI] Only capture a single CUDA graph size in CI by default (#25951)


Signed-off-by: default avatarHarry Mellor <19981378+hmellor@users.noreply.github.com>
parent 1405f0c7
......@@ -731,6 +731,9 @@ class VllmRunner:
init_ctx = (nullcontext() if default_torch_num_threads is None else
set_default_torch_num_threads(default_torch_num_threads))
if not kwargs.get("compilation_config", None):
kwargs["compilation_config"] = {"cudagraph_capture_sizes": [8]}
with init_ctx:
self.llm = LLM(
model=model_name,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment