Unverified Commit 1c63a16b authored by Micah Williamson's avatar Micah Williamson Committed by GitHub
Browse files

[Core] Run garbage collector after CUDA graph capture to fix throughput regression (#24128)


Signed-off-by: default avatarGregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: default avatarGregory Shtrasberg <Gregory.Shtrasberg@amd.com>
parent 922d3b40
......@@ -2885,6 +2885,7 @@ class GPUModelRunner(LoRAModelRunnerMixin, KVConnectorModelRunnerMixin):
finally:
if should_freeze:
gc.unfreeze()
gc.collect()
# Trigger CUDA graph capture for specific shapes.
# Capture the large shapes first so that the smaller shapes
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment