Unverified Commit c5ebe040 authored by Jeremy Reizenstein's avatar Jeremy Reizenstein Committed by GitHub
Browse files

test_attention compat with coming xformers change (#20487)


Co-authored-by: default avatargemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
parent 8d763cb8
...@@ -450,7 +450,8 @@ def test_multi_query_kv_attention( ...@@ -450,7 +450,8 @@ def test_multi_query_kv_attention(
start += seq_len start += seq_len
# xformers.AttentionBias to Tensor for use in reference impl. # xformers.AttentionBias to Tensor for use in reference impl.
alibi_bias = [ alibi_bias = [
b.materialize(b.shape, device=device).squeeze() for b in attn_bias b.materialize((1, num_query_heads, i, i), device=device).squeeze()
for b, i in zip(attn_bias, seq_lens)
] ]
else: else:
attn_bias = BlockDiagonalCausalMask.from_seqlens(seq_lens) attn_bias = BlockDiagonalCausalMask.from_seqlens(seq_lens)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment