GitLab

Implements dual-chunk-flash-attn backend for dual chunk attention with sparse attention support (#11844)

Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>

Signed-off-by: Yong Hoon Shin <yhshin@meta.com>

Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>

[torch.compile] Add torch inductor pass for fusing silu_and_mul with subsequent scaled_fp8_quant operations (#10867)
Signed-off-by: Sage Moore <sage@neuralmagic.com>

Signed-off-by: Tianyuan Wu <Tianyuan.Wu@amd.com>

Signed-off-by: kaixih <kaixih@nvidia.com>

Signed-off-by: DefTruth <qiustudent_r@163.com>

Signed-off-by: lukas.bluebaum <lukas.bluebaum@aleph-alpha.com>

Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>

Signed-off-by: youkaichao <youkaichao@gmail.com>

Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>

Signed-off-by: Pavani Majety <pmajety@nvidia.com>

Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>

Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>

Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>

Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patrick Horn <patrick.horn@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>

Signed-off-by: simon-mo <xmo@berkeley.edu>
Signed-off-by: Lucas Wilkinson <lcwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>

Integrates the block-quantized kernels introduced in
https://github.com/vllm-project/vllm/pull/11868

 for use in linear
layers.
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>

Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Alexander Matveev <59768536+alexm-neuralmagic@users.noreply.github.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>

Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>

Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com>
Co-authored-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

Signed-off-by: ElizaWszola <eliza@neuralmagic.com>

Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>

Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: youkaichao <youkaichao@126.com>

Signed-off-by: Hanzhi Zhou <hanzhi713@gmail.com>

Signed-off-by: youkaichao <youkaichao@gmail.com>

Signed-off-by: charlifu <charlifu@amd.com>