GitLab

Co-authored-by: roy <jasonailu87@gmail.com>

Co-authored-by: zhaoyang <zhao.yang16@zte.com.cn>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

Add a 1-line docstring to explain why calling context_attention_fwd twice in test_prefix_prefill.py (#2553)

Co-authored-by: DouHappy <2278958187@qq.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

Co-authored-by: wangguoya <wangguoya@baidu.com>
Co-authored-by: Yang Zhao <zhaoyangstar@foxmail.com>

Co-authored-by: One <imone@tuta.io>

* [FIX] Fix Alibi implementation in PagedAttention kernel

* Fix test_attention

* Fix

---------
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Oliver-ss <yuansongwx@outlook.com>

Signed-off-by: Tao Peng <jiankeng.pt@alibaba-inc.com>

Co-authored-by: oliveryuan <oliveryuan@basemind.com>

Co-authored-by: woWoosuk Kwon <woosuk.kwon@berkeley.edu>

* optimize

* add benchmark

* add assert

* add test

* Add query stride to multi_query_cached_kv_attention

* Add kernel benchmark script