Commits · 4851c202bbfb5dadc5a3b3b4409d3f083f4e8e90 · OpenDAS / vllm_cscc

11 Sep, 2024 1 commit
- fix run error · 9b902f9e
  zhuwenwen authored Sep 11, 2024
  
  9b902f9e
10 Sep, 2024 3 commits
- [Bugfix] lookahead block table with cuda graph max capture (#8340) · 22f3a4bc
  Alexander Matveev authored Sep 10, 2024
```
[Bugfix] Ensure multistep lookahead allocation is compatible with cuda graph max capture (#8340)
```
  22f3a4bc
- [Spec Decode] Move ops.advance_step to flash attn advance_step (#8224) · 5faedf1b
  Kevin Lin authored Sep 10, 2024
  
  5faedf1b
- update fa interface param · 5a9db327
  zhuwenwen authored Sep 10, 2024
  
  5a9db327
09 Sep, 2024 1 commit
- add cutlass fa and support bloom nn layout · 82f1ffdf
  zhuwenwen authored Sep 09, 2024
  
  82f1ffdf
05 Sep, 2024 1 commit
- [Core/Bugfix] Add query dtype as per FlashInfer API requirements. (#8173) · e39ebf5c
  Elfie Guo authored Sep 04, 2024
  
  e39ebf5c
31 Aug, 2024 1 commit
- [Bugfix] bugfix and add model test for flashinfer fp8 kv cache. (#8013) · 622f8abf
  Pavani Majety authored Aug 30, 2024
  
  622f8abf
30 Aug, 2024 2 commits
- [TPU][Bugfix] Fix tpu type api (#8035) · 2684efc4
  Woosuk Kwon authored Aug 30, 2024
  
  2684efc4
- [TPU] Support single and multi-host TPUs on GKE (#7613) · 2148441f
  Richard Liu authored Aug 30, 2024
  
  2148441f
29 Aug, 2024 2 commits
- [Core][Kernels] Enable FP8 KV Cache with Flashinfer backend. + BugFix for... · 6b342156
  Pavani Majety authored Aug 29, 2024
```
[Core][Kernels] Enable FP8 KV Cache with Flashinfer backend.  + BugFix for kv_cache_dtype=auto (#7985)
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  6b342156
- Revert "[Core][Kernels] Use FlashInfer backend for FP8 KV Cache when available." (#7982) · ef99a787
  youkaichao authored Aug 28, 2024
  
  ef99a787
28 Aug, 2024 1 commit
- [Core][Kernels] Use FlashInfer backend for FP8 KV Cache when available. (#7798) · b98cc28f
  Pavani Majety authored Aug 28, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  b98cc28f
27 Aug, 2024 1 commit
- Revert #7509 (#7887) · 9606c719
  Cody Yu authored Aug 27, 2024
  
  9606c719
21 Aug, 2024 3 commits
- [BUG] fix crash on flashinfer backend with cudagraph disabled, when attention... · 53328d75
  LI MOU authored Aug 21, 2024
```
[BUG] fix crash on flashinfer backend with cudagraph disabled, when attention group_size not in [1,2,4,8] (#7509)
```
  53328d75
- Refactoring the optimized kernel · 0aa2480f
  zhuwenwen authored Aug 21, 2024
  
  0aa2480f
- Add VLLM_USE_PA_PRINT_PARAM flag to print pa size · 2f9e0bad
  zhuwenwen authored Aug 21, 2024
  
  2f9e0bad
20 Aug, 2024 1 commit
- [Core] Add `AttentionState` abstraction (#7663) · 3b682179
  Antoni Baum authored Aug 20, 2024
  
  3b682179
19 Aug, 2024 1 commit
- Add fa pad conditions and automatic switching strategy · b40f2ffc
  zhuwenwen authored Aug 19, 2024
  
  b40f2ffc
16 Aug, 2024 2 commits
- [spec decode] [4/N] Move update_flash_attn_metadata to attn backend (#7571) · f366f633
  William Lin authored Aug 16, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  f366f633
- register custom op for flash attn and use from torch.ops (#7536) · 54bd9a03
  youkaichao authored Aug 15, 2024
  
  54bd9a03
15 Aug, 2024 1 commit
- update triton fa config for head_dim>128 · 8ae929fb
  zhuwenwen authored Aug 15, 2024
  
  8ae929fb
14 Aug, 2024 1 commit
- update the configuration of triton fa · 33290303
  zhuwenwen authored Aug 14, 2024
  
  33290303
13 Aug, 2024 2 commits
- update prefix_prefill params · e658bf7f
  zhuwenwen authored Aug 13, 2024
  
  e658bf7f
- [hardware] unify usage of is_tpu to current_platform.is_tpu() (#7102) · 4d2dc507
  youkaichao authored Aug 13, 2024
  
  4d2dc507
12 Aug, 2024 3 commits
- [Core/Bugfix] Add FP8 K/V Scale and dtype conversion for prefix/prefill Triton Kernel (#7208) · a046f863
  jon-chuang authored Aug 12, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  a046f863
- [Bugfix] Fix logit soft cap in flash-attn backend (#7425) · cfba4def
  Woosuk Kwon authored Aug 12, 2024
  
  cfba4def
- [Kernel] Flashinfer correctness fix for v0.1.3 (#7319) · ec2affa8
  Lily Liu authored Aug 12, 2024
  
  ec2affa8
10 Aug, 2024 1 commit
- add benchmarks to vllm and add env of VLLM_USE_FLASH_ATTN_AUTO · efb2f75f
  zhuwenwen authored Aug 10, 2024
  
  efb2f75f
09 Aug, 2024 2 commits
- [Misc] Add numpy implementation of `compute_slot_mapping` (#7377) · 999ef0b9
  Antoni Baum authored Aug 09, 2024
  
  999ef0b9
- [Performance] Optimize e2e overheads: Reduce python allocations (#7162) · e02ac556
  Alexander Matveev authored Aug 09, 2024
  
  e02ac556
07 Aug, 2024 1 commit
- [Kernel] Fix Flashinfer Correctness (#7284) · e53dfd3e
  Lily Liu authored Aug 07, 2024
  
  e53dfd3e
06 Aug, 2024 1 commit

[Core] Subclass ModelRunner to support cross-attention & encoder sequences... · fd95e026

afeldman-nm authored Aug 06, 2024


[Core] Subclass ModelRunner to support cross-attention & encoder sequences (towards eventual encoder/decoder model support) (#4942)
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>

fd95e026

05 Aug, 2024 1 commit
- [MISC] Use non-blocking transfer in prepare_input (#7172) · ef527be0
  Cody Yu authored Aug 05, 2024
  
  ef527be0
03 Aug, 2024 1 commit
- [Bugfix] Fix block table for seqs that have prefix cache hits (#7018) · fb2c1c86
  Zach Zheng authored Aug 02, 2024
  
  fb2c1c86
02 Aug, 2024 1 commit
- [Kernel] Fix input for flashinfer prefill wrapper. (#7008) · 954f7305
  Lily Liu authored Aug 01, 2024
  
  954f7305
01 Aug, 2024 1 commit
- [Misc] Support attention logits soft-capping with flash-attn (#7022) · 805a8a75
  Woosuk Kwon authored Aug 01, 2024
  
  805a8a75
29 Jul, 2024 1 commit
- [Bugfix] Allow vllm to still work if triton is not installed. (#6786) · 9a7e2d05
  Thomas Parnell authored Jul 29, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  9a7e2d05
27 Jul, 2024 3 commits
- [TPU] Reduce compilation time & Upgrade PyTorch XLA version (#6856) · fad5576c
  Woosuk Kwon authored Jul 27, 2024
  
  fad5576c
- [Hardware][TPU] Implement tensor parallelism with Ray (#5871) · 52f07e3d
  Woosuk Kwon authored Jul 26, 2024
  
  52f07e3d
- [Model] H2O Danube3-4b (#6451) · 14dbd5a7
  Joe authored Jul 26, 2024
  
  14dbd5a7