Commits · afd0da2186c1d58fb48e138df0a2f548612b5d7d · OpenDAS / vllm_cscc · GitLab

01 Feb, 2025 1 commit

[Attention] Deepseek v3 MLA support with FP8 compute (#12601) · baeded25

Lucas Wilkinson authored Feb 01, 2025



This PR implements the Deepseek V3 support by performing matrix absorption the fp8 weights 

---------
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Alexander Matveev <59768536+alexm-neuralmagic@users.noreply.github.com>

baeded25

31 Jan, 2025 2 commits

[Attention] MLA decode optimizations (#12528) · cabaf4ef

Lucas Wilkinson authored Jan 31, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Alexander Matveev <59768536+alexm-neuralmagic@users.noreply.github.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>

cabaf4ef

[ROCm][AMD][Model] llama 3.2 support upstreaming (#12421) · a1fc18c0
Aleksandr Malyshev authored Jan 30, 2025
```
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
```
a1fc18c0

28 Jan, 2025 4 commits
- [Kernel] Pipe attn_logits_soft_cap through paged attention TPU kernels (#12482) · 80fcc3ed
  fenghuizhang authored Jan 28, 2025
```
Signed-off-by: Fenghui Zhang <fhzhang@google.com>
```
  80fcc3ed
- [CI] fix pre-commit error (#12494) · dd66fd2b
  Mengqing Cao authored Jan 28, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  dd66fd2b
- [Neuron][Kernel] NKI-based flash-attention kernel with paged KV cache (#11277) · ddee88d0
  Liangfu Chen authored Jan 27, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
Co-authored-by: Jiangfei Duan <jfduan@outlook.com>
```
  ddee88d0
- Update `pre-commit` hooks (#12475) · 823ab796
  Harry Mellor authored Jan 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  823ab796
27 Jan, 2025 2 commits
- [FlashInfer] Upgrade to 0.2.0 (#11194) · 2bc3fbba
  Bowen Wang authored Jan 28, 2025
```
Signed-off-by: Bowen Wang <abmfy@icloud.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  2bc3fbba
- [Bugfix] Fix missing seq_start_loc in xformers prefill metadata (#12464) · 372bf089
  Isotr0py authored Jan 27, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  372bf089
26 Jan, 2025 3 commits
- [Bugfix/CI] Fix broken kernels/test_mha.py (#12450) · 72f48804
  Tyler Michael Smith authored Jan 26, 2025
  
  72f48804
- [Misc] Revert FA on ViT #12355 and #12435 (#12445) · a5255270
  Roger Wang authored Jan 26, 2025
  
  a5255270
- [Misc][Bugfix] FA3 support to ViT MHA layer (#12435) · 2a0309a6
  Roger Wang authored Jan 25, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  2a0309a6
25 Jan, 2025 1 commit
- [Misc] Add FA2 support to ViT MHA layer (#12355) · f1fc0510
  Isotr0py authored Jan 25, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  f1fc0510
24 Jan, 2025 1 commit
- [Bugfix][Kernel] Fix CUDA 11.8 being broken by FA3 build (#12375) · ab5bbf5a
  Lucas Wilkinson authored Jan 24, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  ab5bbf5a
23 Jan, 2025 2 commits
- [FP8][Kernel] Dynamic kv cache scaling factors computation (#11906) · e97f802b
  Gregory Shtrasberg authored Jan 23, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
```
  e97f802b
- [Kernel] Flash Attention 3 Support (#12093) · 978b45f3
  Lucas Wilkinson authored Jan 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  978b45f3
22 Jan, 2025 1 commit
- [core] separate builder init and builder prepare for each batch (#12253) · 66818e5b
  youkaichao authored Jan 22, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  66818e5b
20 Jan, 2025 1 commit
- [Misc] Pass `attention` to impl backend (#12218) · 86bfb6db
  wangxiyuan authored Jan 20, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  86bfb6db
17 Jan, 2025 1 commit
- [V1] Move more control of kv cache initialization from model_executor to EngineCore (#11960) · 69d765f5
  Chen Zhang authored Jan 17, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  69d765f5
15 Jan, 2025 2 commits
- [Platform] Do not raise error if _Backend is not found (#12023) · 3adf0ffd
  wangxiyuan authored Jan 15, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>
```
  3adf0ffd
- [Misc] Add multipstep chunked-prefill support for FlashInfer (#10467) · 0794e744
  Elfie Guo authored Jan 14, 2025
  
  0794e744
14 Jan, 2025 3 commits
- [Bugfix][Kernel] Give unique name to BlockSparseFlashAttention (#12040) · a2d2acb4
  Chen Zhang authored Jan 14, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  a2d2acb4
- [Platform] Add output for Attention Backend (#11981) · 2e0e0176
  wangxiyuan authored Jan 14, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  2e0e0176
- [Kernel] Revert the API change of Attention.forward (#12038) · 1f18adb2
  Chen Zhang authored Jan 14, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  1f18adb2
13 Jan, 2025 2 commits
- [Kernel] unified_attention for Attention.forward (#11967) · 0f8cafe2
  Chen Zhang authored Jan 13, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  0f8cafe2
- [Bug] Fix usage of `.transpose()` and `.view()` consecutively. (#11979) · 9dd02d85
  Siyuan Li authored Jan 13, 2025
  
  9dd02d85
10 Jan, 2025 1 commit
- [torch.compile] Hide KV cache behind torch.compile boundary (#11677) · cf5f000d
  Chen Zhang authored Jan 10, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  cf5f000d
09 Jan, 2025 2 commits

[platform] Allow platform specify attention backend (#11609) · 405eb8e3

wangxiyuan authored Jan 09, 2025


Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>

405eb8e3

[Misc] Move `print_*_once` from utils to logger (#11298) · d848800e

Cyrus Leung authored Jan 09, 2025

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
Co-authored-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>

d848800e

06 Jan, 2025 2 commits
- [Kernel] Move attn_type to Attention.__init__() (#11690) · e20c92bb
  Chen Zhang authored Jan 07, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  e20c92bb
- [Doc][2/N] Reorganize Models and Usage sections (#11755) · ee77fdb5
  Cyrus Leung authored Jan 06, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ee77fdb5
02 Jan, 2025 1 commit
- add softcap interface for gemma-2 · f9b567df
  zhuwenwen authored Jan 02, 2025
  
  f9b567df
30 Dec, 2024 1 commit
- fix interface error · 02c3e313
  zhuwenwen authored Dec 30, 2024
  
  02c3e313
24 Dec, 2024 1 commit
- [attn][tiny fix] fix attn backend in MultiHeadAttention (#11463) · 5c796324
  Mengqing Cao authored Dec 24, 2024
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  5c796324
23 Dec, 2024 1 commit
- [Docs] Convert rST to MyST (Markdown) (#11145) · 32aa2059
  Rafael Vasquez authored Dec 23, 2024
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  32aa2059
17 Dec, 2024 1 commit
- [Misc] Allow passing logits_soft_cap for xformers backend (#11252) · f9ecbb18
  Isotr0py authored Dec 17, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  f9ecbb18
13 Dec, 2024 2 commits
- [Bugfix][Hardware][CPU] Enable Gemma2 with SDPA on CPU backend (#11169) · 0a56bcc0
  Jani Monoses authored Dec 13, 2024
  
  0a56bcc0
- remove ck fa interface · 54ddee7f
  zhuwenwen authored Dec 13, 2024
  
  54ddee7f
11 Dec, 2024 2 commits
- [Doc] Update docs to refer to pooling models (#11093) · cad5c0a6
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cad5c0a6
- [feat]优化medusa代码，通过VLLM_TREE_DECODING环境变量控制是否采用tree-style解码，计算逻辑主干隔离 · 44e3ca68
  王敏 authored Dec 11, 2024
  
  44e3ca68