Commits · 1c3ffdbeccaf75f7b885b1700a4011fb4f63a7ab · OpenDAS / vllm_cscc

21 Sep, 2025 1 commit
- [V0 Deprecation] Remove V0 sampling metadata (#25345) · 1c3ffdbe
  Woosuk Kwon authored Sep 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
  1c3ffdbe
18 Sep, 2025 1 commit
- [Spec Decode] Efficient padded speculation (#24539) · b7433ca1
  Benjamin Chislett authored Sep 18, 2025
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
  b7433ca1
16 Sep, 2025 1 commit

[Core/DBO][1/N] Add Dual-Batch Overlap mechanism to VLLM (#23693) · 56793995

Sage Moore authored Sep 16, 2025


Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

56793995

11 Sep, 2025 1 commit
- Add the support for the qwen3 next model (a hybrid attention model). (#24526) · e93f4cc9
  Tao He authored Sep 11, 2025
```
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  e93f4cc9
01 Sep, 2025 1 commit
- [bugfix]fix MTP hidden states (#24056) · a344a5aa
  Lucia Fang authored Sep 01, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
```
  a344a5aa
20 Aug, 2025 3 commits
- [FIXBUG ] Allow disabling rocm_aiter_fa backend for ROCm GPUs not compatible with AITER (#22795) · 3b11b26b
  JartX authored Aug 20, 2025
```
Signed-off-by: JartX <sagformas@epdcenter.es>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  3b11b26b
- [Misc] Add max_seq_len to CommonAttentionMetadata (#23216) · d6d13bd4
  Woosuk Kwon authored Aug 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  d6d13bd4
- [Model][V1] Support Ernie MTP (#22169) · 7cd17e22
  xyxinyang authored Aug 20, 2025
```
Signed-off-by: zhouchong <zhouchong03@baidu.com>
Co-authored-by: zhouchong <zhouchong03@baidu.com>
```
  7cd17e22
13 Aug, 2025 1 commit
- [V1] Add tree drafting tests for eagle spec decoding (#22705) · d94e3026
  Giancarlo Delfin authored Aug 13, 2025
```
Signed-off-by: Giancarlo Delfin <gdelfin@meta.com>
```
  d94e3026
08 Aug, 2025 1 commit
- [ROCm] [V1] [SpecDec] Enable Speculative Decoding on ROCm V1 Engine (#21496) · 1ee5ead5
  TJian authored Aug 07, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  1ee5ead5
07 Aug, 2025 1 commit

[Attention] Support multiple attention metadata builders per kv_cache_spec +... · 1dc8a70b

Lucas Wilkinson authored Aug 06, 2025


[Attention] Support multiple attention metadata builders per kv_cache_spec  + proper local attention no hybrid kv cache fix (#21588)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

1dc8a70b

04 Aug, 2025 1 commit
- Add tree attention backend for v1 (part 1) (#20401) · aa7012eb
  Giancarlo Delfin authored Aug 03, 2025
```
Signed-off-by: Giancarlo Delfin <gdelfin@meta.com>
```
  aa7012eb
31 Jul, 2025 1 commit
- [Meta] Official Eagle mm support, first enablement on llama4 (#20788) · 9e0726e5
  zhiweiz authored Jul 31, 2025
```
Signed-off-by: morgendave <morgendave@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.me>
```
  9e0726e5
26 Jul, 2025 1 commit

Support encoder-only models without KV-Cache (#21270) · 1cd6eaba

Maximilien de Bayser authored Jul 26, 2025


Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>

1cd6eaba

17 Jul, 2025 1 commit
- [Attention] Refactor attention metadata builder interface (#20466) · 76b49444
  Lucas Wilkinson authored Jul 17, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  76b49444
06 Jul, 2025 1 commit
- [V1] Support any head size for FlexAttention backend (#20467) · 9fb52e52
  Cyrus Leung authored Jul 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9fb52e52
25 Jun, 2025 1 commit
- [V1][Speculative Decoding] Fix DeepSeek MTP (#20022) · 8359f4c8
  cjackal authored Jun 26, 2025
```
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
```
  8359f4c8
13 Jun, 2025 3 commits
- [CUDA] Enable full cudagraph for FlashMLA (#18581) · 3597b06a
  Luka Govedič authored Jun 13, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  3597b06a
- [torch.compile] reorganize the cache directory to support compiling multiple models (#19064) · d70bc7c0
  youkaichao authored Jun 13, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  d70bc7c0
- [Bugfix] Fix EAGLE vocab embedding for multimodal target model (#19570) · c68698b3
  qizixi authored Jun 12, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  c68698b3
06 Jun, 2025 1 commit
- [Bugfix] Fix EAGLE vocab embedding construction for Llama 70B (#19033) · 3465b87e
  Benjamin Chislett authored Jun 05, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
  3465b87e
03 Jun, 2025 1 commit
- [Misc] Add SPDX-FileCopyrightText (#19100) · 02f0c7b2
  Simon Mo authored Jun 03, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  02f0c7b2
28 May, 2025 1 commit
- [V1] [Bugfix] eagle bugfix and enable correct lm_head for multimodal (2) (#18781) · 5e13c07d
  RonaldBXu authored May 27, 2025
```
Signed-off-by: Ronald Xu <ronaldxu@amazon.com>
```
  5e13c07d
24 May, 2025 2 commits
- [V1][Spec Decode] Support multi-layer eagle draft model (#18030) · c1e4a405
  qizixi authored May 24, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  c1e4a405
- [V1][Spec Decode] Small refactors to improve eagle bookkeeping performance (#18424) · d55e446d
  qizixi authored May 23, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  d55e446d
23 May, 2025 4 commits
- [V1] Support Deepseek MTP (#18435) · 2628a69e
  Jiayi Yao authored May 23, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Signed-off-by: YaoJiayi <120040070@link.cuhk.edu.cn>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>
```
  2628a69e
- Revert "[V1] [Bugfix] eagle bugfix and enable correct lm_head for multimodal (#18034)" (#18600) · fbb13a2c
  Cyrus Leung authored May 23, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  fbb13a2c
- [V1] [Bugfix] eagle bugfix and enable correct lm_head for multimodal (#18034) · 4c611348
  RonaldBXu authored May 23, 2025
```
Signed-off-by: Ronald Xu <ronaldxu@amazon.com>
```
  4c611348
- [V1][Spec Decoding] Use model_loader.get_model() to load models (#18273) · c6b636f9
  Mark McLoughlin authored May 23, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  c6b636f9
22 May, 2025 1 commit
- [V1][Spec Decode][Bugfix] Load quantize weights for EAGLE (#18290) · a04720bc
  Ekagra Ranjan authored May 22, 2025
  
  a04720bc
14 May, 2025 1 commit

[V1][Spec Decode] Share input embedding of target model with EAGLE draft model... · 418d2f8b

Ekagra Ranjan authored May 14, 2025


[V1][Spec Decode] Share input embedding of target model with EAGLE draft model to free ~1GB for llama 3 model (#17326)
Co-authored-by: root <root@ekagra-8xh100.us-east5-a.c.serving-efficiency-poc.internal>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

418d2f8b

12 May, 2025 1 commit
- [V1][Spec Decode] Eagle unit tests (#17350) · dc990536
  wwl2755 authored May 12, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  dc990536
07 May, 2025 2 commits
- [Misc] Split model loader (#17712) · 822de7fb
  Jee Jee Li authored May 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  822de7fb
- [BugFix][Spec Decode] Fix hidden size mismatch between target and eagle head (#17740) · 8d84d836
  Woosuk Kwon authored May 06, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  8d84d836
06 May, 2025 2 commits
- [v1] AttentionMetadata for each layer (#17394) · cba31c47
  Chen Zhang authored May 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  cba31c47
- [Bugfix] Fix triton import with local TritonPlaceholder (#17446) · f9bc5a06
  Mengqing Cao authored May 06, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  f9bc5a06
01 May, 2025 1 commit
- [V1][Spec Decode] Apply torch.compile & cudagraph to EAGLE3 (#17504) · 39c0813a
  qizixi authored May 01, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  39c0813a
29 Apr, 2025 2 commits
- [V1][Spec Decode] Apply torch.compile & cudagraph to EAGLE (#17211) · 70788bdb
  Bryan Lu authored Apr 29, 2025
```
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
```
  70788bdb
- [V1][Spec Decode] Make Eagle model arch config driven (#17323) · e1360005
  Ekagra Ranjan authored Apr 28, 2025
  
  e1360005
25 Apr, 2025 1 commit

[V1][Spec Decode] EAGLE-3 Support (#16937) · a0e619e6

Benjamin Chislett authored Apr 25, 2025


Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Co-authored-by: Bryan Lu <yuzhelu@amazon.com>

a0e619e6