Skip to content

GitLab

Unverified Commit 67073dde authored by brayden-hai's avatar brayden-hai Committed by GitHub
Browse files

Garbage collector regression in the online server (#10621)

parent 9a5c42f9
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
python/sglang/srt/model_executor/cuda_graph_runner.py
View file @ 67073dde
...@@ -100,6 +100,7 @@ def freeze_gc(enable_cudagraph_gc: bool): ...@@ -100,6 +100,7 @@ def freeze_gc(enable_cudagraph_gc: bool):
finally: finally:
if should_freeze: if should_freeze:
gc.unfreeze() gc.unfreeze()
gc.collect()
def _to_torch(model: torch.nn.Module, reverse: bool, num_tokens: int): def _to_torch(model: torch.nn.Module, reverse: bool, num_tokens: int):
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment