Commits · 4fbda0b20cc539f72314375c2abc6100ebac8392 · OpenDAS / vllm_cscc

20 Aug, 2025 2 commits

[Feature] use --eplb_config to set eplb param (#20562) · 4fbda0b2

rongfu.leng authored Aug 21, 2025


Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

4fbda0b2

[Misc] Add max_seq_len to CommonAttentionMetadata (#23216) · d6d13bd4
Woosuk Kwon authored Aug 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
d6d13bd4

19 Aug, 2025 3 commits
- [Misc] Fix seq_lens for graph capture (#23175) · 40f26734
  Woosuk Kwon authored Aug 19, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  40f26734
- [Misc] Avoid accessing req_ids inside a loop (#23159) · 21bcc826
  Woosuk Kwon authored Aug 19, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  21bcc826
- [Spec Decode] Make `propose_draft_token_ids` non-blocking for lower TTFT (#23041) · c9b38be8
  Woosuk Kwon authored Aug 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  c9b38be8
18 Aug, 2025 3 commits
- [Misc] Minor refactoring for prepare_inputs (#23116) · 0dd3f4f5
  Woosuk Kwon authored Aug 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  0dd3f4f5
- [Refactor] Define MultiModalKwargsItems separate from MultiModalKwargs (#23053) · 27e8d1ea
  Cyrus Leung authored Aug 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  27e8d1ea
- [Misc] Minor code cleanup for _get_prompt_logprobs_dict (#23064) · 8ea0c275
  Woosuk Kwon authored Aug 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  8ea0c275
17 Aug, 2025 1 commit
- [XPU] fix xpu to set cudagraph batch sizes (#23044) · 21e39436
  Calvin Chen authored Aug 18, 2025
```
Signed-off-by: calvin chen <wen.chen@dynamia.ai>
```
  21e39436
16 Aug, 2025 2 commits

[V1] Logits processors extensibility (#19912) · bf7f470b

afeldman-nm authored Aug 16, 2025


Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Signed-off-by: Andrew Feldman <afeld2012@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

bf7f470b

[Refactor] Allow optional MultiModalKwargsItem in IPC (#23022) · 4dff91c9
Cyrus Leung authored Aug 16, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
4dff91c9

15 Aug, 2025 6 commits

[Structured Outputs] [Bug] Fix misalignment in apply_grammar_bitmask causing... · 1fc375dc

rishitdholakia13 authored Aug 15, 2025


[Structured Outputs] [Bug] Fix misalignment in apply_grammar_bitmask causing unintended masking and NaN logits (#22963)
Signed-off-by: rishitdholakia13 <rishit+github@cohere.com>

1fc375dc

[Log] Debug Once for Randomizing dummy data for DP Rank (#22860) · df5afa82
Wentao Ye authored Aug 15, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
df5afa82
[FIXBUG] Correctly Apply Grammar Bitmask in Mixed Batches (#22896) · 68af77e5
JartX authored Aug 15, 2025
```
Signed-off-by: JartX <sagformas@epdcenter.es>
```
68af77e5

[Core] Allow full cudagraph with separate attention routines and orthogonal to... · 74f441f4

fhl2000 authored Aug 15, 2025


[Core] Allow full cudagraph with separate attention routines and orthogonal to compilation, add support for FA2 and FlashInfer (#20059)
Signed-off-by: fhl <2410591650@qq.com>
Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>

74f441f4

[V1] [Hybrid] Support using float32 for state in Hybrid Models (Mamba2, Mamba1, Minimax) (#22928) · 75531a6c

Thomas Parnell authored Aug 15, 2025


Signed-off-by: Daniel Afrimi <danielafrimi8@gmail.com>
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Daniel Afrimi <danielafrimi8@gmail.com>
Co-authored-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>

75531a6c

[MM] Allow skipping memory profiling for multimodal models. (#22950) · 49252cf5

Roger Wang authored Aug 15, 2025


Signed-off-by: Roger Wang <hey@rogerw.me>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

49252cf5

14 Aug, 2025 1 commit
- [Perf] Dont create unnecessary pooling params (#22876) · 829b9a62
  Lucas Wilkinson authored Aug 14, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  829b9a62
13 Aug, 2025 1 commit
- [Core] Use individual MM items in P0/P1 cache and model runner (#22570) · 19b927e5
  Cyrus Leung authored Aug 13, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  19b927e5
12 Aug, 2025 2 commits

Add: `SupportsEagle3` interface for explicit EAGLE3 support (#22642) · 5a4b4b37
Rahul Tuli authored Aug 12, 2025
```
Signed-off-by: Rahul Tuli <rtuli@redhat.com>
```
5a4b4b37

[Bugfix] Fix ModernBert load & Enable sliding window attention for... · 6d729c43

wang.yuqi authored Aug 12, 2025


[Bugfix] Fix ModernBert load & Enable sliding window attention for bidirectional attention. (#22637)
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>

6d729c43

11 Aug, 2025 2 commits
- [Model] Pooling models default to using chunked prefill & prefix caching if supported. (#20930) · 84cf78ac
  wang.yuqi authored Aug 12, 2025
```
Signed-off-by: wang.yuqi <noooop@126.com>
```
  84cf78ac
- Support token_type_ids in V1 with less code changes (#21985) · 39052dbc
  Maximilien de Bayser authored Aug 11, 2025
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  39052dbc
09 Aug, 2025 1 commit

Implicit language-model-only mode via limit-mm-per-prompt (#22299) · 08b751ba

Roger Wang authored Aug 08, 2025


Signed-off-by: Roger Wang <hey@rogerw.me>
Signed-off-by: Andy Xie <andy.xning@gmail.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: XIn Li <xinli@nvidia.com>
Signed-off-by: Junhao Li <junhao@ubicloud.com>
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
Signed-off-by: zitian zhao <zitian.zhao@tencentmusic.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Signed-off-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Signed-off-by: Linkun <github@lkchen.net>
Co-authored-by: Ning Xie <andy.xning@gmail.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Andrew Sansom <andrew@protopia.ai>
Co-authored-by: Zhiyu <zhiyuc@nvidia.com>
Co-authored-by: Shu Wang <shuw@nvidia.com>
Co-authored-by: XIn Li <xinli@nvidia.com>
Co-authored-by: Junhao Li <streaver91@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Yuxuan Zhang <2448370773@qq.com>
Co-authored-by: ZiTian Zhao <zitian.zhao@tencentmusic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Hong Hanh <hanh.usth@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: lkchen <github@lkchen.net>

08b751ba

07 Aug, 2025 1 commit

[Attention] Support multiple attention metadata builders per kv_cache_spec +... · 1dc8a70b

Lucas Wilkinson authored Aug 06, 2025


[Attention] Support multiple attention metadata builders per kv_cache_spec  + proper local attention no hybrid kv cache fix (#21588)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

1dc8a70b

06 Aug, 2025 1 commit
- [Perf] Parallelize fill_bitmask to accelerate high-throughput guided decoding (#21862) · 7e6544c7
  Benjamin Chislett authored Aug 05, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
  7e6544c7
05 Aug, 2025 1 commit
- [Core] Factor out common logic for MM budget calculation (#22228) · 811ac13d
  Cyrus Leung authored Aug 05, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  811ac13d
03 Aug, 2025 1 commit

[V1] [P/D] Refactor KV Connector Path (#21980) · aefeea0f

David Ben-David authored Aug 03, 2025


Signed-off-by: David Ben-David <davidb@pliops.com>
Co-authored-by: David Ben-David <davidb@pliops.com>

aefeea0f

02 Aug, 2025 2 commits
- [V1] [Hybrid] Validate compatibility of attention backend batch reordering at init time (#21557) · 4abfd879
  Thomas Parnell authored Aug 02, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  4abfd879
- [V1][CUDA] Full cudagraph support for FlashInfer (#21367) · 23322431
  fhl2000 authored Aug 02, 2025
  
  23322431
31 Jul, 2025 1 commit
- [Meta] Official Eagle mm support, first enablement on llama4 (#20788) · 9e0726e5
  zhiweiz authored Jul 31, 2025
```
Signed-off-by: morgendave <morgendave@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.me>
```
  9e0726e5
30 Jul, 2025 2 commits
- Override attention metadata for fast prefill in some KV sharing setups (#21590) · ad510309
  Yong Hoon Shin authored Jul 30, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  ad510309
- [v1][attention] Support Hybrid Allocator + FlashInfer (#21412) · 555e7225
  Chen Zhang authored Jul 29, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  555e7225
29 Jul, 2025 1 commit
- [Bugfix] VLLM_V1 supports passing other compilation levels (#19340) · 04e38500
  Richard Zou authored Jul 29, 2025
```
Signed-off-by: Richard Zou <zou3519@gmail.com>
```
  04e38500
28 Jul, 2025 5 commits
- Revert "[V1] Exception Handling when Loading KV Cache from Remote Store" (#21778) · b18b417f
  Kuntai Du authored Jul 28, 2025
```
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
```
  b18b417f
- [v1][mamba] Added mamba_type into MambaSpec (#21715) · a6c05028
  Asaf Joseph Gardin authored Jul 28, 2025
```
Signed-off-by: asafg <asafg@ai21.com>
Co-authored-by: asafg <asafg@ai21.com>
```
  a6c05028
- [BugFix] Fix ChunkedLocalAttention when the hybrid kv-cache is disabled (#21707) · 139a7f07
  Lucas Wilkinson authored Jul 28, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  139a7f07
- [V1] Exception Handling when Loading KV Cache from Remote Store (#21534) · 15a72ac4
  Adeline authored Jul 28, 2025
```
Signed-off-by: liuyumoye <adeline_ly2023@outlook.com>
Co-authored-by: liuyumoye <adeline_ly2023@outlook.com>
```
  15a72ac4
- [Deprecation][2/N] Replace `--task` with `--runner` and `--convert` (#21470) · 86ae693f
  Cyrus Leung authored Jul 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  86ae693f
26 Jul, 2025 1 commit

Support encoder-only models without KV-Cache (#21270) · 1cd6eaba

Maximilien de Bayser authored Jul 26, 2025


Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>

1cd6eaba