Commits · ab5bbf5ae32bc438803ced4c2a021bb161e00050 · OpenDAS / vllm_cscc · GitLab

24 Jan, 2025 1 commit
- [Bugfix][Kernel] Fix CUDA 11.8 being broken by FA3 build (#12375) · ab5bbf5a
  Lucas Wilkinson authored Jan 24, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  ab5bbf5a
23 Jan, 2025 2 commits
- [FP8][Kernel] Dynamic kv cache scaling factors computation (#11906) · e97f802b
  Gregory Shtrasberg authored Jan 23, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
```
  e97f802b
- [Kernel] Flash Attention 3 Support (#12093) · 978b45f3
  Lucas Wilkinson authored Jan 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  978b45f3
22 Jan, 2025 1 commit
- [core] separate builder init and builder prepare for each batch (#12253) · 66818e5b
  youkaichao authored Jan 22, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  66818e5b
20 Jan, 2025 1 commit
- [Misc] Pass `attention` to impl backend (#12218) · 86bfb6db
  wangxiyuan authored Jan 20, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  86bfb6db
14 Jan, 2025 1 commit
- [Platform] Add output for Attention Backend (#11981) · 2e0e0176
  wangxiyuan authored Jan 14, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  2e0e0176
06 Jan, 2025 1 commit
- [Kernel] Move attn_type to Attention.__init__() (#11690) · e20c92bb
  Chen Zhang authored Jan 07, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  e20c92bb
02 Dec, 2024 1 commit
- [misc] use out argument for flash attention (#10822) · a4c4daf3
  youkaichao authored Dec 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  a4c4daf3
22 Nov, 2024 1 commit
- [torch.compile] support all attention backends (#10558) · eebad39f
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eebad39f
07 Nov, 2024 1 commit
- [Feature] [Spec decode]: Combine chunked prefill with speculative decoding (#9291) · 9d43afcc
  Nicolò Lucchesi authored Nov 07, 2024
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  9d43afcc
04 Nov, 2024 1 commit
- [Misc] Compute query_start_loc/seq_start_loc on CPU (#9447) · 4dbcbbeb
  Yang Zheng authored Nov 04, 2024
```
Co-authored-by: Yang Zheng(SW)(Alex) <you@example.com>
```
  4dbcbbeb
02 Nov, 2024 1 commit
- [Encoder Decoder] Add flash_attn kernel support for encoder-decoder models (#9559) · a78dd330
  sroy745 authored Nov 01, 2024
  
  a78dd330
01 Nov, 2024 2 commits
- [Core][VLM] Add precise multi-modal placeholder tracking (#8346) · 6c0b7f54
  Peter Salas authored Nov 01, 2024
```
Signed-off-by: Peter Salas <peter@fixie.ai>
```
  6c0b7f54
- [torch.compile] directly register custom op (#9896) · 96e0c9cb
  youkaichao authored Oct 31, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  96e0c9cb
21 Oct, 2024 1 commit
- [Doc] Consistent naming of attention backends (#9498) · 496e991d
  Thomas Parnell authored Oct 21, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  496e991d
20 Oct, 2024 1 commit
- [Kernel] Support sliding window in flash attention backend (#9403) · 4fa3e333
  Chen Zhang authored Oct 20, 2024
  
  4fa3e333
17 Oct, 2024 1 commit

[Core] Deprecating block manager v1 and make block manager v2 default (#8704) · 81ede99c

Kuntai Du authored Oct 17, 2024

Removing the block manager v1. This is the initial piece of prefix-caching-centric design. In order to achieve prefix-caching-centric design, we need to simplify the code path so that we only use v2 block manager (which has much higher performance on prefix caching).

81ede99c

12 Oct, 2024 1 commit
- [SpecDec] Remove Batch Expansion (2/3) (#9298) · 89feb4c8
  Lily Liu authored Oct 11, 2024
  
  89feb4c8
03 Oct, 2024 1 commit
- [misc] add forward context for attention (#9029) · 9aaf14c6
  youkaichao authored Oct 03, 2024
  
  9aaf14c6
02 Oct, 2024 1 commit
- [Core] CUDA Graphs for Multi-Step + Chunked-Prefill (#8645) · afb050b2
  Varun Sundar Rabindranath authored Oct 02, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  afb050b2
01 Oct, 2024 1 commit
- [Spec Decode] (1/2) Remove batch expansion (#8839) · 15702038
  Lily Liu authored Oct 01, 2024
  
  15702038
27 Sep, 2024 2 commits
- [Core] Multi-Step + Single Step Prefills via Chunked Prefill code path (#8378) · c2ec430a
  Varun Sundar Rabindranath authored Sep 27, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  c2ec430a
- [torch.compile] use empty tensor instead of None for profiling (#8875) · a9b15c60
  youkaichao authored Sep 27, 2024
  
  a9b15c60
21 Sep, 2024 1 commit
- [Kernel] Build flash-attn from source (#8245) · 71c60491
  Luka Govedič authored Sep 21, 2024
  
  71c60491
12 Sep, 2024 2 commits
- [multi-step] add flashinfer backend (#7928) · a6c0f365
  William Lin authored Sep 12, 2024
  
  a6c0f365
- [torch.compile] hide slicing under custom op for inductor (#8384) · 7de49aa8
  youkaichao authored Sep 12, 2024
  
  7de49aa8
10 Sep, 2024 2 commits
- [Bugfix] lookahead block table with cuda graph max capture (#8340) · 22f3a4bc
  Alexander Matveev authored Sep 10, 2024
```
[Bugfix] Ensure multistep lookahead allocation is compatible with cuda graph max capture (#8340)
```
  22f3a4bc
- [Spec Decode] Move ops.advance_step to flash attn advance_step (#8224) · 5faedf1b
  Kevin Lin authored Sep 10, 2024
  
  5faedf1b
20 Aug, 2024 1 commit
- [Core] Add `AttentionState` abstraction (#7663) · 3b682179
  Antoni Baum authored Aug 20, 2024
  
  3b682179
16 Aug, 2024 2 commits
- [spec decode] [4/N] Move update_flash_attn_metadata to attn backend (#7571) · f366f633
  William Lin authored Aug 16, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  f366f633
- register custom op for flash attn and use from torch.ops (#7536) · 54bd9a03
  youkaichao authored Aug 15, 2024
  
  54bd9a03
12 Aug, 2024 1 commit
- [Bugfix] Fix logit soft cap in flash-attn backend (#7425) · cfba4def
  Woosuk Kwon authored Aug 12, 2024
  
  cfba4def
09 Aug, 2024 1 commit
- [Performance] Optimize e2e overheads: Reduce python allocations (#7162) · e02ac556
  Alexander Matveev authored Aug 09, 2024
  
  e02ac556
05 Aug, 2024 1 commit
- [MISC] Use non-blocking transfer in prepare_input (#7172) · ef527be0
  Cody Yu authored Aug 05, 2024
  
  ef527be0
03 Aug, 2024 1 commit
- [Bugfix] Fix block table for seqs that have prefix cache hits (#7018) · fb2c1c86
  Zach Zheng authored Aug 02, 2024
  
  fb2c1c86
01 Aug, 2024 1 commit
- [Misc] Support attention logits soft-capping with flash-attn (#7022) · 805a8a75
  Woosuk Kwon authored Aug 01, 2024
  
  805a8a75
25 Jul, 2024 1 commit
- [Bugfix] Fix decode tokens w. CUDA graph (#6757) · 309aaef8
  Cody Yu authored Jul 24, 2024
  
  309aaef8
24 Jul, 2024 1 commit
- Add fp8 support to `reshape_and_cache_flash` (#6667) · 0e63494c
  Antoni Baum authored Jul 24, 2024
  
  0e63494c
23 Jul, 2024 1 commit
- [Core] Modulize prepare input and attention metadata builder (#6596) · e0c15758
  Cody Yu authored Jul 22, 2024
  
  e0c15758
20 Jul, 2024 1 commit
- [Misc] Consolidate and optimize logic for building padded tensors (#6541) · 9042d683
  Cyrus Leung authored Jul 20, 2024
  
  9042d683