Unverified Commit bc72e5bd authored by Yineng Zhang's avatar Yineng Zhang Committed by GitHub
Browse files

add cuda graph capture failure possible solution (#3430)

parent 014cab4d
...@@ -237,6 +237,7 @@ class CudaGraphRunner: ...@@ -237,6 +237,7 @@ class CudaGraphRunner:
"1. disable cuda graph by --disable-cuda-graph\n" "1. disable cuda graph by --disable-cuda-graph\n"
"2. set --mem-fraction-static to a smaller value (e.g., 0.8 or 0.7)\n" "2. set --mem-fraction-static to a smaller value (e.g., 0.8 or 0.7)\n"
"3. disable torch compile by not using --enable-torch-compile\n" "3. disable torch compile by not using --enable-torch-compile\n"
"4. set --cuda-graph-max-bs to a smaller value (e.g., 32)\n"
"Open an issue on GitHub https://github.com/sgl-project/sglang/issues/new/choose \n" "Open an issue on GitHub https://github.com/sgl-project/sglang/issues/new/choose \n"
) )
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment