Commits · 16484d394c6046890635b7895edb4deca2f1b1b0 · OpenDAS / vllm_cscc

17 Dec, 2025 1 commit

[Core][MM] Optimize encoder cache manager by operating with embeddings only (#30475) · 16484d39

Roger Wang authored Dec 16, 2025


Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Sun Kim <sunytokki@gmail.com>
(cherry picked from commit f5f51e59)

16484d39

10 Dec, 2025 1 commit

[P/D] KV Load Failure Recovery/Abort Configuration (#26813) · a9e4106f

Will Eaton authored Dec 10, 2025


Signed-off-by: Will Eaton <weaton@redhat.com>
Signed-off-by: Will Eaton <me@wseaton.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: chaunceyjiang <chaunceyjiang@gmail.com>

a9e4106f

02 Dec, 2025 2 commits

[Bugfix] fix --scheduling-policy=priority & n>1 crashes engine (#29764) · 0a9caca9

Chauncey authored Dec 03, 2025


Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

0a9caca9

[Core] Support reseting all running requests' KV while calling `reset_prefix_cache` (#28827) · d0cd7289

Zhuohan Li authored Dec 01, 2025


Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

d0cd7289

21 Nov, 2025 1 commit
- [KV Connector] Fix async connector prefix cache metrics (#28585) · c6fa3895
  Mark McLoughlin authored Nov 21, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
```
  c6fa3895
16 Nov, 2025 1 commit
- [Model] Allow users to control skip reading cache per request. (#28194) · a55b6463
  wang.yuqi authored Nov 16, 2025
```
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
```
  a55b6463
05 Nov, 2025 1 commit
- [Feature]: Add corrupted request metric to V1 metrics system. (#27306) · e1560178
  Snehlata authored Nov 06, 2025
```
Signed-off-by: atalhens <sneh.lata@nutanix.com>
```
  e1560178
14 Oct, 2025 1 commit
- [Core] Streamline some structured output related code (#26737) · 4aed506b
  Nick Hill authored Oct 14, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  4aed506b
12 Oct, 2025 1 commit
- Update `Optional[x]` -> `x | None` and `Union[x, y]` to `x | y` (#26633) · 8fcaaf6a
  Harry Mellor authored Oct 12, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8fcaaf6a
05 Oct, 2025 1 commit
- Convert formatting to use `ruff` instead of `yapf` + `isort` (#26247) · d6953beb
  Harry Mellor authored Oct 05, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6953beb
27 Sep, 2025 1 commit
- [Core] Don't count preempted tokens in prefix cache hit rate (#25787) · 8bf8f458
  Zhuohan Li authored Sep 26, 2025
```
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
```
  8bf8f458
19 Sep, 2025 1 commit

[CORE] Prompt Embeddings Support for v1 Engine (#24278) · 9a4600e4

Andrew Sansom authored Sep 18, 2025


Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: Andrew Sansom <qthequartermasterman@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

9a4600e4

18 Sep, 2025 1 commit

[Chore] Cleanup guided namespace, move to structured outputs config (#22772) · 29283e89

Aaron Pham authored Sep 18, 2025

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

29283e89

12 Sep, 2025 2 commits

[Multimodal] Remove legacy multimodal fields in favor of MultiModalFeatureSpec (#24548) · 0377802c
Flora Feng authored Sep 12, 2025
```
Signed-off-by: sfeng33 <4florafeng@gmail.com>
```
0377802c

[V1] feat:add engine v1 tracing (#20372) · 40b6c912

RichardoMu authored Sep 12, 2025


Signed-off-by: Mu Huai <tianbowen.tbw@antgroup.com>
Signed-off-by: Ye Zhang <zhysishu@gmail.com>
Signed-off-by: RichardoMu <44485717+RichardoMrMu@users.noreply.github.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Mu Huai <tianbowen.tbw@antgroup.com>
Co-authored-by: Ye Zhang <zhysishu@gmail.com>
Co-authored-by: Benjamin Bartels <benjamin@bartels.dev>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: 瑜琮 <ly186375@antfin.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>

40b6c912

29 Aug, 2025 1 commit
- [Multimodal] Consolidate mm inputs into MultiModalFeatureSpec (#23779) · 69f46359
  Flora Feng authored Aug 29, 2025
```
Signed-off-by: sfeng33 <4florafeng@gmail.com>
```
  69f46359
17 Aug, 2025 1 commit
- [Misc] Convert use_structured_output property into constant (#23060) · 6d243efe
  Woosuk Kwon authored Aug 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6d243efe
16 Aug, 2025 1 commit
- [Refactor] Allow optional MultiModalKwargsItem in IPC (#23022) · 4dff91c9
  Cyrus Leung authored Aug 16, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  4dff91c9
15 Aug, 2025 1 commit
- [v1] Move block_hashes from KVCacheManager to Request.block_hashes (#19728) · c280066f
  Or Ozeri authored Aug 16, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  c280066f
13 Aug, 2025 1 commit
- [Core] Use individual MM items in P0/P1 cache and model runner (#22570) · 19b927e5
  Cyrus Leung authored Aug 13, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  19b927e5
15 Jul, 2025 1 commit
- Implement Async Scheduling (#19970) · d4d30940
  Woosuk Kwon authored Jul 14, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  d4d30940
23 Jun, 2025 1 commit
- [Core] feat: Implement Priority Scheduling in V1 Engine (#19057) · 4a0f7888
  amit authored Jun 23, 2025
```
Signed-off-by: amit <amit.man@gmail.com>
Co-authored-by: Roger Wang <Rogerw0108@gmail.com>
```
  4a0f7888
20 Jun, 2025 1 commit
- Export NaNs in logits to scheduler_stats if output is corrupted (#18777) · 2e3e3c86
  Vlad Tiberiu Mihailescu authored Jun 20, 2025
```
Signed-off-by: Vlad Mihailescu <vtmihailescu@gmail.com>
```
  2e3e3c86
19 Jun, 2025 1 commit

Support embedding models in V1 (#16188) · 799397ee

Maximilien de Bayser authored Jun 19, 2025


Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>

799397ee

18 Jun, 2025 1 commit
- [Misc] Add __str__ for RequestStatus (#19780) · d4629dc4
  lkchen authored Jun 17, 2025
```
Signed-off-by: Linkun Chen <github@lkchen.net>
```
  d4629dc4
03 Jun, 2025 1 commit
- [Misc] Add SPDX-FileCopyrightText (#19100) · 02f0c7b2
  Simon Mo authored Jun 03, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  02f0c7b2
30 May, 2025 1 commit
- [Perf] API-server scaleout with many-to-many server-engine comms (#17546) · 2dbe8c07
  Nick Hill authored May 30, 2025
  
  2dbe8c07
23 May, 2025 1 commit
- [Feature][V1]: suupports cached_tokens in response usage (#18149) · b046cf79
  Chauncey authored May 23, 2025
```
Co-authored-by: simon-mo <xmo@berkeley.edu>
```
  b046cf79
15 May, 2025 1 commit

fix: typos (#18151) · a9944aab

omahs authored May 15, 2025


Signed-off-by: omahs <73983677+omahs@users.noreply.github.com>

a9944aab

14 May, 2025 1 commit
- [KVConnector] Keep KVTransferParams as a dict (#18033) · 59dd311c
  Nick Hill authored May 14, 2025
  
  59dd311c
12 May, 2025 1 commit

[P/D] NIXL Integration (#17751) · d1911020

Robert Shaw authored May 12, 2025


Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Brent Salisbury <bsalisbu@redhat.com>

d1911020

30 Apr, 2025 1 commit
- [Core] Prevent side-channel attacks via cache salting (#17045) · 77073c77
  Marko Rosenmueller authored Apr 30, 2025
```
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
```
  77073c77
26 Apr, 2025 1 commit
- [Core] Remove prompt string from engine core data structures (#17214) · df6f3ce8
  Nick Hill authored Apr 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  df6f3ce8
09 Apr, 2025 1 commit
- [Bugfix] Avoid transferring cached multi-modal items from P0 to P1 (#16273) · e484e028
  Cyrus Leung authored Apr 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e484e028
08 Apr, 2025 1 commit

[V1] Scatter and gather placeholders in the model runner (#16076) · f2ebb6f5

Roger Wang authored Apr 07, 2025


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>

f2ebb6f5

04 Apr, 2025 2 commits

Revert "[V1] Scatter and gather placeholders in the model runner" (#16075) · af51d80f
Roger Wang authored Apr 04, 2025

af51d80f

[V1] Scatter and gather placeholders in the model runner (#15712) · f5722a50

Cyrus Leung authored Apr 05, 2025


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

f5722a50

30 Mar, 2025 1 commit
- [V1][Scheduler] Avoid calling `_try_schedule_encoder_inputs` for every request (#15778) · 9b459eca
  Woosuk Kwon authored Mar 30, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  9b459eca
29 Mar, 2025 1 commit
- [Misc][V1] Misc code streamlining (#15723) · 6d531ad7
  Nick Hill authored Mar 28, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  6d531ad7
14 Mar, 2025 1 commit
- [V1][Minor] Minor code cleanup for scheduling metrics (#14800) · c77620d2
  Woosuk Kwon authored Mar 14, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  c77620d2