Commits · acf8aeb79e23c32217dd37b5e96847302ae4d0b7 · OpenDAS / vllm_cscc

07 Aug, 2025 1 commit

[Attention] Support multiple attention metadata builders per kv_cache_spec +... · 1dc8a70b

Lucas Wilkinson authored Aug 06, 2025


[Attention] Support multiple attention metadata builders per kv_cache_spec  + proper local attention no hybrid kv cache fix (#21588)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

1dc8a70b

04 Aug, 2025 1 commit
- Add tree attention backend for v1 (part 1) (#20401) · aa7012eb
  Giancarlo Delfin authored Aug 03, 2025
```
Signed-off-by: Giancarlo Delfin <gdelfin@meta.com>
```
  aa7012eb
31 Jul, 2025 1 commit
- [Meta] Official Eagle mm support, first enablement on llama4 (#20788) · 9e0726e5
  zhiweiz authored Jul 31, 2025
```
Signed-off-by: morgendave <morgendave@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.me>
```
  9e0726e5
26 Jul, 2025 1 commit

Support encoder-only models without KV-Cache (#21270) · 1cd6eaba

Maximilien de Bayser authored Jul 26, 2025


Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>

1cd6eaba

17 Jul, 2025 1 commit
- [Attention] Refactor attention metadata builder interface (#20466) · 76b49444
  Lucas Wilkinson authored Jul 17, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  76b49444
06 Jul, 2025 1 commit
- [V1] Support any head size for FlexAttention backend (#20467) · 9fb52e52
  Cyrus Leung authored Jul 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9fb52e52
25 Jun, 2025 1 commit
- [V1][Speculative Decoding] Fix DeepSeek MTP (#20022) · 8359f4c8
  cjackal authored Jun 26, 2025
```
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
```
  8359f4c8
13 Jun, 2025 3 commits
- [CUDA] Enable full cudagraph for FlashMLA (#18581) · 3597b06a
  Luka Govedič authored Jun 13, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  3597b06a
- [torch.compile] reorganize the cache directory to support compiling multiple models (#19064) · d70bc7c0
  youkaichao authored Jun 13, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  d70bc7c0
- [Bugfix] Fix EAGLE vocab embedding for multimodal target model (#19570) · c68698b3
  qizixi authored Jun 12, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  c68698b3
06 Jun, 2025 1 commit
- [Bugfix] Fix EAGLE vocab embedding construction for Llama 70B (#19033) · 3465b87e
  Benjamin Chislett authored Jun 05, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
  3465b87e
03 Jun, 2025 1 commit
- [Misc] Add SPDX-FileCopyrightText (#19100) · 02f0c7b2
  Simon Mo authored Jun 03, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  02f0c7b2
28 May, 2025 1 commit
- [V1] [Bugfix] eagle bugfix and enable correct lm_head for multimodal (2) (#18781) · 5e13c07d
  RonaldBXu authored May 27, 2025
```
Signed-off-by: Ronald Xu <ronaldxu@amazon.com>
```
  5e13c07d
24 May, 2025 2 commits
- [V1][Spec Decode] Support multi-layer eagle draft model (#18030) · c1e4a405
  qizixi authored May 24, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  c1e4a405
- [V1][Spec Decode] Small refactors to improve eagle bookkeeping performance (#18424) · d55e446d
  qizixi authored May 23, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  d55e446d
23 May, 2025 4 commits
- [V1] Support Deepseek MTP (#18435) · 2628a69e
  Jiayi Yao authored May 23, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Signed-off-by: YaoJiayi <120040070@link.cuhk.edu.cn>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>
```
  2628a69e
- Revert "[V1] [Bugfix] eagle bugfix and enable correct lm_head for multimodal (#18034)" (#18600) · fbb13a2c
  Cyrus Leung authored May 23, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  fbb13a2c
- [V1] [Bugfix] eagle bugfix and enable correct lm_head for multimodal (#18034) · 4c611348
  RonaldBXu authored May 23, 2025
```
Signed-off-by: Ronald Xu <ronaldxu@amazon.com>
```
  4c611348
- [V1][Spec Decoding] Use model_loader.get_model() to load models (#18273) · c6b636f9
  Mark McLoughlin authored May 23, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  c6b636f9
22 May, 2025 1 commit
- [V1][Spec Decode][Bugfix] Load quantize weights for EAGLE (#18290) · a04720bc
  Ekagra Ranjan authored May 22, 2025
  
  a04720bc
14 May, 2025 1 commit

[V1][Spec Decode] Share input embedding of target model with EAGLE draft model... · 418d2f8b

Ekagra Ranjan authored May 14, 2025


[V1][Spec Decode] Share input embedding of target model with EAGLE draft model to free ~1GB for llama 3 model (#17326)
Co-authored-by: root <root@ekagra-8xh100.us-east5-a.c.serving-efficiency-poc.internal>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

418d2f8b

12 May, 2025 1 commit
- [V1][Spec Decode] Eagle unit tests (#17350) · dc990536
  wwl2755 authored May 12, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  dc990536
07 May, 2025 2 commits
- [Misc] Split model loader (#17712) · 822de7fb
  Jee Jee Li authored May 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  822de7fb
- [BugFix][Spec Decode] Fix hidden size mismatch between target and eagle head (#17740) · 8d84d836
  Woosuk Kwon authored May 06, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  8d84d836
06 May, 2025 2 commits
- [v1] AttentionMetadata for each layer (#17394) · cba31c47
  Chen Zhang authored May 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  cba31c47
- [Bugfix] Fix triton import with local TritonPlaceholder (#17446) · f9bc5a06
  Mengqing Cao authored May 06, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  f9bc5a06
01 May, 2025 1 commit
- [V1][Spec Decode] Apply torch.compile & cudagraph to EAGLE3 (#17504) · 39c0813a
  qizixi authored May 01, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  39c0813a
29 Apr, 2025 2 commits
- [V1][Spec Decode] Apply torch.compile & cudagraph to EAGLE (#17211) · 70788bdb
  Bryan Lu authored Apr 29, 2025
```
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
```
  70788bdb
- [V1][Spec Decode] Make Eagle model arch config driven (#17323) · e1360005
  Ekagra Ranjan authored Apr 28, 2025
  
  e1360005
25 Apr, 2025 1 commit

[V1][Spec Decode] EAGLE-3 Support (#16937) · a0e619e6

Benjamin Chislett authored Apr 25, 2025


Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Co-authored-by: Bryan Lu <yuzhelu@amazon.com>

a0e619e6

23 Apr, 2025 1 commit
- [V1][Spec Decode] Always use argmax for sampling draft tokens (#16899) · 41fb013d
  Woosuk Kwon authored Apr 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  41fb013d
22 Apr, 2025 1 commit
- [BugFix][Spec Decode] No in-place update to draft probs (#16952) · 1311913f
  Woosuk Kwon authored Apr 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  1311913f
21 Apr, 2025 1 commit
- [V1][Spec Decode] Handle draft tokens beyond max_model_len (#16087) · 3a0fba5c
  Woosuk Kwon authored Apr 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3a0fba5c
10 Apr, 2025 1 commit
- [V1][Spec Decode] Eagle Model loading (#16035) · e8224f3d
  Lily Liu authored Apr 10, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  e8224f3d
02 Apr, 2025 1 commit
- [Spec Decode] Fix input triton kernel for eagle (#15909) · 24b7fb45
  Ekagra Ranjan authored Apr 01, 2025
  
  24b7fb45
01 Apr, 2025 1 commit
- [V1][Spec Decode] Implement Eagle Proposer [1/N] (#15729) · e75a6301
  Woosuk Kwon authored Apr 01, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e75a6301