Unverified Commit 6e956d9e authored by Woosuk Kwon's avatar Woosuk Kwon Committed by GitHub
Browse files

[Model Runner V2] Add dummy profile_cudagraph_memory API (#36520)


Signed-off-by: default avatarWoosuk Kwon <woosuk@inferact.ai>
parent 1e0f917b
......@@ -473,6 +473,10 @@ class GPUModelRunner(LoRAModelRunnerMixin):
# SP is not supported yet.
return num_scheduled_tokens
def profile_cudagraph_memory(self) -> int:
# NOTE(woosuk): It is TBD whether we keep this API or not.
return 0
@torch.inference_mode()
def capture_model(self) -> int:
if not self.cudagraph_manager.needs_capture():
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment