Skip to content

GitLab

Unverified Commit 86a2c473 authored by Lianmin Zheng's avatar Lianmin Zheng Committed by GitHub
Browse files

[Fix] Fix seq_lens_sum for cuda graph runner in padded cases (#1789)

parent 1701b0db
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
python/sglang/srt/model_executor/cuda_graph_runner.py
View file @ 86a2c473
...@@ -307,7 +307,7 @@ class CudaGraphRunner: ...@@ -307,7 +307,7 @@ class CudaGraphRunner:
bs, bs,
self.req_pool_indices, self.req_pool_indices,
self.seq_lens, self.seq_lens,
forward_batch.seq_lens_sum, forward_batch.seq_lens_sum + (bs - raw_bs),
self.encoder_lens, self.encoder_lens,
) )
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment