Unverified Commit f88763ef authored by Ignacio Sica's avatar Ignacio Sica Committed by GitHub
Browse files

[Bugfix] add seq_lens_cpu_upper_bound to CommonAttentionMetadata in mla_runner.py (#40844)


Signed-off-by: default avatarignaciosica <mignacio.sica@gmail.com>
parent 333529de
...@@ -404,6 +404,7 @@ def _build_attention_metadata( ...@@ -404,6 +404,7 @@ def _build_attention_metadata(
query_start_loc=q_start_gpu, query_start_loc=q_start_gpu,
query_start_loc_cpu=q_start_cpu, query_start_loc_cpu=q_start_cpu,
seq_lens=seq_lens_gpu, seq_lens=seq_lens_gpu,
seq_lens_cpu_upper_bound=seq_lens_cpu,
_seq_lens_cpu=seq_lens_cpu, _seq_lens_cpu=seq_lens_cpu,
_num_computed_tokens_cpu=num_computed_tokens_cpu, _num_computed_tokens_cpu=num_computed_tokens_cpu,
slot_mapping=slot_mapping, slot_mapping=slot_mapping,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment