Unverified Commit 17000d2b authored by AinL's avatar AinL Committed by GitHub
Browse files

Remove Unintended Capture Batch Sizes in AMD HIP Graph Runner (#4638)

parent 668ecc6c
......@@ -124,8 +124,8 @@ def get_batch_sizes_to_capture(model_runner: ModelRunner):
# capture less.
capture_bs = list(range(1, 9)) + list(range(9, 33, 2)) + [64, 96, 128, 160]
if _is_hip:
capture_bs += [i * 8 for i in range(21, 33)]
if _is_hip:
capture_bs += [i * 8 for i in range(21, 33)]
if max(capture_bs) > model_runner.req_to_token_pool.size:
# In some case (e.g., with a small GPU or --max-running-requests), the #max-running-requests
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment