Commits · d4d309409f2396e68e4b5a67ede194913502388b · OpenDAS / vllm_cscc · GitLab

15 Jul, 2025 1 commit
- Implement Async Scheduling (#19970) · d4d30940
  Woosuk Kwon authored Jul 14, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  d4d30940
14 Jul, 2025 1 commit
- [Prefix Cache] Add reproducible prefix-cache block hashing using SHA-256 + CBOR (64bit) (#20511) · 66f6fbd3
  Maroon Ayoub authored Jul 14, 2025
```
Signed-off-by: Maroon Ayoub <maroon.ayoub@ibm.com>
```
  66f6fbd3
12 Jul, 2025 1 commit
- [Sched] Enhance the logic to remove stopped requests from queues (#20739) · f45a3328
  Woosuk Kwon authored Jul 12, 2025
  
  f45a3328
04 Jul, 2025 1 commit
- [Structured Outputs][V1] Skipping with models doesn't contain tokenizers (#20365) · 4a98edff
  Aaron Pham authored Jul 04, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  4a98edff
30 Jun, 2025 1 commit
- [Optimization] Use Shared `CachedRequestData` Instance Across All Requests (#20232) · 2863befc
  Woosuk Kwon authored Jun 30, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2863befc
23 Jun, 2025 1 commit
- [Core] feat: Implement Priority Scheduling in V1 Engine (#19057) · 4a0f7888
  amit authored Jun 23, 2025
```
Signed-off-by: amit <amit.man@gmail.com>
Co-authored-by: Roger Wang <Rogerw0108@gmail.com>
```
  4a0f7888
19 Jun, 2025 1 commit

Support embedding models in V1 (#16188) · 799397ee

Maximilien de Bayser authored Jun 19, 2025


Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>

799397ee

12 Jun, 2025 1 commit
- [Bugfix] Respect num-gpu-blocks-override in v1 (#19503) · c9280e63
  jmswen authored Jun 12, 2025
```
Signed-off-by: Jon Swenson <jmswen@gmail.com>
```
  c9280e63
10 Jun, 2025 1 commit
- [Core] Use tuple for kv cache group block ids (#19175) · 646d62f6
  Nick Hill authored Jun 09, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  646d62f6
06 Jun, 2025 1 commit
- [v1] Hybrid Memory Allocator (#17996) · f8a1a2d1
  Chen Zhang authored Jun 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  f8a1a2d1
04 Jun, 2025 1 commit

[Bugfix] Max concurrency estimation and check_enough_kv_cache_memory for... · a8da78ea

Chen Zhang authored Jun 04, 2025


[Bugfix] Max concurrency estimation and check_enough_kv_cache_memory for models with sliding window layers (#19029)
Signed-off-by: Chen Zhang <zhangch99@outlook.com>

a8da78ea

03 Jun, 2025 3 commits
- [Bugfix] get_num_blocks_to_allocate with null_block (#19031) · b5fd9506
  Chen Zhang authored Jun 04, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  b5fd9506
- [Misc] Add SPDX-FileCopyrightText (#19100) · 02f0c7b2
  Simon Mo authored Jun 03, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  02f0c7b2
- [v1][KVCacheManager] Rename BlockHashType to BlockHash (#19015) · f32fcd94
  Chen Zhang authored Jun 03, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  f32fcd94
30 May, 2025 1 commit
- [Perf] API-server scaleout with many-to-many server-engine comms (#17546) · 2dbe8c07
  Nick Hill authored May 30, 2025
  
  2dbe8c07
23 May, 2025 2 commits
- [v1] Redo "Support multiple KV cache groups in GPU model runner (#17945)" (#18593) · 6550114c
  Chen Zhang authored May 24, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  6550114c
- [Feature][V1]: suupports cached_tokens in response usage (#18149) · b046cf79
  Chauncey authored May 23, 2025
```
Co-authored-by: simon-mo <xmo@berkeley.edu>
```
  b046cf79
21 May, 2025 1 commit
- Revert "[v1] Support multiple KV cache groups in GPU model runner (#17945) (#18459) · bb0a3112
  Mark McLoughlin authored May 21, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  bb0a3112
15 May, 2025 2 commits
- [UT] Add ut for none hash (#17892) · 420caf75
  Ning Xie authored May 15, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  420caf75
- [v1] Support multiple KV cache groups in GPU model runner (#17945) · e60f550b
  Chen Zhang authored May 15, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  e60f550b
14 May, 2025 1 commit
- [v1][KVCacheManager] pass num_new_computed_tokens to kv cache manager (#18001) · f2ae883b
  Chen Zhang authored May 14, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  f2ae883b
13 May, 2025 1 commit
- [v1][KVCacheManager] Avoid full cache hit by controlling max_length (#17999) · f0d610a8
  Chen Zhang authored May 13, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f0d610a8
12 May, 2025 1 commit

[P/D] NIXL Integration (#17751) · d1911020

Robert Shaw authored May 12, 2025


Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Brent Salisbury <bsalisbu@redhat.com>

d1911020

10 May, 2025 1 commit
- [v1] Rename specialized_manager.py to single_type_kv_cache_manager.py (#17946) · ca66a167
  Chen Zhang authored May 11, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  ca66a167
09 May, 2025 2 commits
- [v1] Move block management logic from KVCacheManager to SpecializedManager (#17474) · 200da9a5
  Chen Zhang authored May 09, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  200da9a5
- [BUGFIX]: return fast when request requires prompt logprobs (#17251) · d310e6de
  Ning Xie authored May 09, 2025
  
  d310e6de
06 May, 2025 1 commit
- [v1] Introduce KVCacheBlocks as interface between Scheduler and KVCacheManager (#17479) · aabcd2ca
  Chen Zhang authored May 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  aabcd2ca
04 May, 2025 1 commit
- Add full API docs and improve the UX of navigating them (#17485) · d6484ef3
  Harry Mellor authored May 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6484ef3
02 May, 2025 1 commit
- [BugFix] Fix Memory Leak (#17567) · c777df79
  Robert Shaw authored May 02, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  c777df79
30 Apr, 2025 3 commits
- [v1][Spec Decode] Make sliding window compatible with eagle prefix caching (#17398) · 81ecf425
  Chen Zhang authored May 01, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  81ecf425
- [V1][Metrics] add support for kv event publishing (#16750) · 0be6d05b
  Alec authored Apr 30, 2025
```
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
```
  0be6d05b
- [Core] Prevent side-channel attacks via cache salting (#17045) · 77073c77
  Marko Rosenmueller authored Apr 30, 2025
```
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
```
  77073c77
27 Apr, 2025 1 commit
- [V1][Spec Decode] Make eagle compatible with prefix caching. (#17137) · 20e489ea
  Lily Liu authored Apr 27, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  20e489ea
26 Apr, 2025 2 commits
- [MISC] rename interval to max_recent_requests (#14285) · fd11a325
  Ning Xie authored Apr 27, 2025
  
  fd11a325
- [Core] Remove prompt string from engine core data structures (#17214) · df6f3ce8
  Nick Hill authored Apr 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  df6f3ce8
24 Apr, 2025 2 commits
- [V1][Spec Decoding] Add num_drafts and num_accepted_tokens_per_position metrics (#16665) · 340d7b1b
  Mark McLoughlin authored Apr 24, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  340d7b1b
- [V1][PP] Optimization: continue scheduling prefill chunks (#17080) · c0dfd975
  Rui Qiao authored Apr 24, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  c0dfd975
22 Apr, 2025 1 commit
- [V1] Remove pre-allocation for KV cache (#16941) · c4ab9f3e
  Woosuk Kwon authored Apr 22, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  c4ab9f3e
21 Apr, 2025 1 commit
- [V1][Spec Decode] Handle draft tokens beyond max_model_len (#16087) · 3a0fba5c
  Woosuk Kwon authored Apr 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3a0fba5c
19 Apr, 2025 1 commit
- [V1][Misc] stop update prefix cache stats when logs_stats is disabled (#16460) · d9737ca1
  vie-serendipity authored Apr 19, 2025
```
Signed-off-by: vie-serendipity <2733147505@qq.com>
```
  d9737ca1