Unverified Commit a114bf20 authored by SnowCharm's avatar SnowCharm Committed by GitHub
Browse files

[Perf] Optimize `_update_states` for GPU model runner (#16910)


Signed-off-by: default avatarsnowcharm <snowcharmqq@gmail.com>
parent 3097ce3a
......@@ -454,7 +454,7 @@ class GPUModelRunner(LoRAModelRunnerMixin):
# Add the new or resumed requests to the persistent batch.
# The smaller empty indices are filled first.
removed_req_indices = sorted(removed_req_indices, reverse=True)
removed_req_indices.sort(reverse=True)
for req_id in req_ids_to_add:
req_state = self.requests[req_id]
if removed_req_indices:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment