Unverified Commit 6b8f66ef authored by Yineng Zhang's avatar Yineng Zhang Committed by GitHub
Browse files

misc: update cuda graph capture exception log (#894)

parent 7937a886
...@@ -312,10 +312,12 @@ class ModelRunner: ...@@ -312,10 +312,12 @@ class ModelRunner:
self.cuda_graph_runner.capture(batch_size_list) self.cuda_graph_runner.capture(batch_size_list)
except RuntimeError as e: except RuntimeError as e:
raise Exception( raise Exception(
f"Capture cuda graph failed: {e}. Possible solutions:\n" f"Capture cuda graph failed: {e}\n"
f"1. disable cuda graph by --disable-cuda-graph\n" "Possible solutions:\n"
f"2. set --mem-fraction-static to a smaller value\n" "1. disable torch compile by not using --enable-torch-compile\n"
f"Open an issue on GitHub with reproducible scripts if you need help.\n" "2. disable cuda graph by --disable-cuda-graph\n"
"3. set --mem-fraction-static to a smaller value\n"
"Open an issue on GitHub https://github.com/sgl-project/sglang/issues/new/choose \n"
) )
@torch.inference_mode() @torch.inference_mode()
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment