vllm/attention/backends/torch_sdpa.py · 8e192ff967b44b186ea02d30e49fddf656fdfe50 · OpenDAS / vllm_cscc

[Kernel][Backend][Model] Blocksparse flash attention kernel and Phi-3-Small model (#4799) · 8e192ff9

Eric Xihui Lin authored May 25, 2024


Co-authored-by: beagleski <yunanzhang@microsoft.com>
Co-authored-by: bapatra <bapatra@microsoft.com>
Co-authored-by: Barun Patra <codedecde@users.noreply.github.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

8e192ff9

torch_sdpa.py 10.2 KB

Replace torch_sdpa.py