Skip to content

GitLab

Unverified Commit 679c3bca authored by Lianmin Zheng's avatar Lianmin Zheng Committed by GitHub
Browse files

Fix typo in cuda_graph_bs (#2813)

parent 656aed58
Hide whitespace changes
Inline Side-by-side
Showing with 0 additions and 5 deletions
python/sglang/srt/model_executor/cuda_graph_runner.py
View file @ 679c3bca
...@@ -131,11 +131,6 @@ class CudaGraphRunner: ...@@ -131,11 +131,6 @@ class CudaGraphRunner:
else: else:
self.capture_bs = [1, 2, 4] + [i * 8 for i in range(1, 21)] self.capture_bs = [1, 2, 4] + [i * 8 for i in range(1, 21)]
if model_runner.server_args.disable_cuda_graph_padding:
self.capture_bs = list(range(1, 33)) + [64, 128]
else:
self.capture_bs = [1, 2, 4] + [i * 8 for i in range(1, 21)]
if max(self.capture_bs) > model_runner.req_to_token_pool.size: if max(self.capture_bs) > model_runner.req_to_token_pool.size:
# In some case (e.g., with a small GPU or --max-running-requests), the #max-running-requests # In some case (e.g., with a small GPU or --max-running-requests), the #max-running-requests
# is very samll. We add more values here to make sure we capture the maximum bs. # is very samll. We add more values here to make sure we capture the maximum bs.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment