Commits · 201768d595388bb3cf24c2bf9f2519c136068f85 · OpenDAS / vllm_cscc

27 Nov, 2024 1 commit
- 1.优化medusa推理，节省cpu耗时 · 28375803
  王敏 authored Nov 27, 2024
```
2.更新medusa readme
3.解决benchmark_moe报错问题
```
  28375803
18 Nov, 2024 1 commit
- [fix]解决test_model_input中报的找不到move_cache错误 · 4d29e0a8
  王敏 authored Nov 18, 2024
  
  4d29e0a8
24 Oct, 2024 1 commit
- 增加medusa并行解码功能,后续增加使用说明和测试文档 · 19bc93d9
  王敏 authored Oct 24, 2024
  
  19bc93d9
18 Oct, 2024 1 commit
- 删除v2 8192限制 · a7e3c9d6
  zhangshao authored Oct 18, 2024
  
  a7e3c9d6
17 Oct, 2024 1 commit
- add VLLM_USE_TC_PAGED_ATTN to convert tc pa · f164171b
  zhuwenwen authored Oct 17, 2024
  
  f164171b
16 Oct, 2024 1 commit
- update paged_attn.py · 85e8224c
  zhuwenwen authored Oct 16, 2024
  
  85e8224c
11 Oct, 2024 1 commit
- [Model] support telechat_12B · 6d4b207a
  zhuwenwen authored Oct 11, 2024
  
  6d4b207a
27 Sep, 2024 1 commit
- update version and deps · 5a9c236d
  zhuwenwen authored Sep 27, 2024
  
  5a9c236d
24 Sep, 2024 1 commit
- 优化pa小batch性能（pa_v2），优化pa小seq性能(pa_v1)，reusekv=16优化 · 2b91ac93
  zhuwenwen authored Sep 24, 2024
  
  2b91ac93
21 Sep, 2024 1 commit
- [Kernel] Build flash-attn from source (#8245) · 71c60491
  Luka Govedič authored Sep 21, 2024
  
  71c60491
20 Sep, 2024 3 commits
- add pa tc · 0c70376b
  zhuwenwen authored Sep 20, 2024
  
  0c70376b
- choose ck or cutlass implementation based on the fa version · 7fafc730
  zhuwenwen authored Sep 20, 2024
  
  7fafc730
- [bugfix] [AMD] add multi-step advance_step to ROCmFlashAttentionMetadata (#8474) · 9e5ec35b
  William Lin authored Sep 19, 2024
  
  9e5ec35b
19 Sep, 2024 1 commit
- [Kernel][Amd] Add fp8 kv cache support for rocm custom paged attention (#8577) · 9cc373f3
  Charlie Fu authored Sep 19, 2024
  
  9cc373f3
18 Sep, 2024 3 commits
- [CI/Build] Update Ruff version (#8469) · 9d104b5b
  Aaron Pham authored Sep 18, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  9d104b5b
- [CI/Build] Avoid CUDA initialization (#8534) · 6ffa3f31
  Cyrus Leung authored Sep 18, 2024
  
  6ffa3f31
- update platforms init · bc84829b
  zhuwenwen authored Sep 18, 2024
  
  bc84829b
17 Sep, 2024 1 commit
- [Encoder decoder] Add cuda graph support during decoding for encoder-decoder models (#7631) · 1009e93c
  sroy745 authored Sep 17, 2024
  
  1009e93c
14 Sep, 2024 1 commit
- [Kernel][Hardware][Amd]Custom paged attention kernel for rocm (#8310) · 1ef0d2ef
  Charlie Fu authored Sep 13, 2024
  
  1ef0d2ef
13 Sep, 2024 2 commits
- [Hardware][intel GPU] bump up ipex version to 2.3 (#8365) · 85172520
  Kunshang Ji authored Sep 14, 2024
```
Co-authored-by: Yan Ma <yan.ma@intel.com>
```
  85172520
- remove the automatic switching strategy of fa · 408f663a
  zhuwenwen authored Sep 13, 2024
  
  408f663a
12 Sep, 2024 3 commits
- [Bugfix] multi-step + flashinfer: ensure cuda graph compatible (#8427) · 01987725
  Alexander Matveev authored Sep 12, 2024
  
  01987725
- [multi-step] add flashinfer backend (#7928) · a6c0f365
  William Lin authored Sep 12, 2024
  
  a6c0f365
- [torch.compile] hide slicing under custom op for inductor (#8384) · 7de49aa8
  youkaichao authored Sep 12, 2024
  
  7de49aa8
11 Sep, 2024 1 commit
- fix run error · 9b902f9e
  zhuwenwen authored Sep 11, 2024
  
  9b902f9e
10 Sep, 2024 3 commits
- [Bugfix] lookahead block table with cuda graph max capture (#8340) · 22f3a4bc
  Alexander Matveev authored Sep 10, 2024
```
[Bugfix] Ensure multistep lookahead allocation is compatible with cuda graph max capture (#8340)
```
  22f3a4bc
- [Spec Decode] Move ops.advance_step to flash attn advance_step (#8224) · 5faedf1b
  Kevin Lin authored Sep 10, 2024
  
  5faedf1b
- update fa interface param · 5a9db327
  zhuwenwen authored Sep 10, 2024
  
  5a9db327
09 Sep, 2024 1 commit
- add cutlass fa and support bloom nn layout · 82f1ffdf
  zhuwenwen authored Sep 09, 2024
  
  82f1ffdf
05 Sep, 2024 1 commit
- [Core/Bugfix] Add query dtype as per FlashInfer API requirements. (#8173) · e39ebf5c
  Elfie Guo authored Sep 04, 2024
  
  e39ebf5c
31 Aug, 2024 1 commit
- [Bugfix] bugfix and add model test for flashinfer fp8 kv cache. (#8013) · 622f8abf
  Pavani Majety authored Aug 30, 2024
  
  622f8abf
30 Aug, 2024 2 commits
- [TPU][Bugfix] Fix tpu type api (#8035) · 2684efc4
  Woosuk Kwon authored Aug 30, 2024
  
  2684efc4
- [TPU] Support single and multi-host TPUs on GKE (#7613) · 2148441f
  Richard Liu authored Aug 30, 2024
  
  2148441f
29 Aug, 2024 2 commits
- [Core][Kernels] Enable FP8 KV Cache with Flashinfer backend. + BugFix for... · 6b342156
  Pavani Majety authored Aug 29, 2024
```
[Core][Kernels] Enable FP8 KV Cache with Flashinfer backend.  + BugFix for kv_cache_dtype=auto (#7985)
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  6b342156
- Revert "[Core][Kernels] Use FlashInfer backend for FP8 KV Cache when available." (#7982) · ef99a787
  youkaichao authored Aug 28, 2024
  
  ef99a787
28 Aug, 2024 1 commit
- [Core][Kernels] Use FlashInfer backend for FP8 KV Cache when available. (#7798) · b98cc28f
  Pavani Majety authored Aug 28, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  b98cc28f
27 Aug, 2024 1 commit
- Revert #7509 (#7887) · 9606c719
  Cody Yu authored Aug 27, 2024
  
  9606c719
21 Aug, 2024 3 commits
- [BUG] fix crash on flashinfer backend with cudagraph disabled, when attention... · 53328d75
  LI MOU authored Aug 21, 2024
```
[BUG] fix crash on flashinfer backend with cudagraph disabled, when attention group_size not in [1,2,4,8] (#7509)
```
  53328d75
- Refactoring the optimized kernel · 0aa2480f
  zhuwenwen authored Aug 21, 2024
  
  0aa2480f
- Add VLLM_USE_PA_PRINT_PARAM flag to print pa size · 2f9e0bad
  zhuwenwen authored Aug 21, 2024
  
  2f9e0bad