Commits · 998eea4a0eaceb40bb3597c0d40ce2cf729dd1fa · OpenDAS / vllm_cscc

07 May, 2025 1 commit
- [Frontend] Add missing chat templates for various MLLMs (#17758) · 8a15c260
  Cyrus Leung authored May 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8a15c260
06 May, 2025 1 commit
- [v1] Introduce KVCacheBlocks as interface between Scheduler and KVCacheManager (#17479) · aabcd2ca
  Chen Zhang authored May 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  aabcd2ca
05 May, 2025 3 commits
- [TPU][V1] Add support for top-logprobs (#17072) · 5941e0b7
  Nicolò Lucchesi authored May 05, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  5941e0b7
- [TPU] Enable gemma3-27b with TP>1 on multi-chips. (#17335) · 97659408
  XiongfeiWei authored May 05, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  97659408
- [BugFix] Increase timeout for startup failure test (#17642) · 5ea5c514
  Nick Hill authored May 05, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  5ea5c514
04 May, 2025 1 commit
- Add full API docs and improve the UX of navigating them (#17485) · d6484ef3
  Harry Mellor authored May 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6484ef3
02 May, 2025 1 commit
- [BugFix] Fix Memory Leak (#17567) · c777df79
  Robert Shaw authored May 02, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  c777df79
01 May, 2025 3 commits
- [CI][Bugfix] Fix failing V1 Test due to missing 'cache_salt' arg (#17500) · aa4502e7
  Michael Goin authored Apr 30, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  aa4502e7
- [CI][TPU] Skip structured outputs+spec decode tests on TPU (#17510) · 17b4d85f
  Michael Goin authored Apr 30, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  17b4d85f
- [CI][TPU] Skip Multimodal test (#17488) · dbc18e78
  Siyuan Liu authored Apr 30, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  dbc18e78
30 Apr, 2025 6 commits
- [v1][Spec Decode] Make sliding window compatible with eagle prefix caching (#17398) · 81ecf425
  Chen Zhang authored May 01, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  81ecf425
- [V1] Allow turning off pickle fallback in vllm.v1.serial_utils (#17427) · 947f2f53
  Russell Bryant authored Apr 30, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  947f2f53
- [V1][Metrics] add support for kv event publishing (#16750) · 0be6d05b
  Alec authored Apr 30, 2025
```
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
```
  0be6d05b
- [Core] Prevent side-channel attacks via cache salting (#17045) · 77073c77
  Marko Rosenmueller authored Apr 30, 2025
```
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
```
  77073c77
- [TPU][V1][CI] Update regression test baseline for v6 CI (#17064) · a7d5b016
  Nicolò Lucchesi authored Apr 30, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  a7d5b016
- [V1][Feature] Enable Speculative Decoding with Structured Outputs (#14702) · 34120f5a
  Benjamin Chislett authored Apr 29, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
```
  34120f5a
29 Apr, 2025 4 commits
- Remove Bamba 9B from CI (#17407) · 7489ec0b
  Harry Mellor authored Apr 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  7489ec0b
- Simplify (and fix) passing of guided decoding backend options (#17008) · a6977dbd
  Harry Mellor authored Apr 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a6977dbd
- [Misc]Use a platform independent interface to obtain the device attributes (#17100) · bdb2cdda
  ponix-j authored Apr 29, 2025
  
  bdb2cdda
- implement Structural Tag with Guidance backend (#17333) · 86d9fc29
  Michał Moskal authored Apr 28, 2025
```
Signed-off-by: Michal Moskal <michal@moskal.me>
```
  86d9fc29
27 Apr, 2025 1 commit
- [V1][Spec Decode] Make eagle compatible with prefix caching. (#17137) · 20e489ea
  Lily Liu authored Apr 27, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  20e489ea
26 Apr, 2025 6 commits
- [MISC] rename interval to max_recent_requests (#14285) · fd11a325
  Ning Xie authored Apr 27, 2025
  
  fd11a325
- [V1] Add `structural_tag` support using xgrammar (#17085) · f8acd01f
  Russell Bryant authored Apr 26, 2025
  
  f8acd01f
- [Core] Remove prompt string from engine core data structures (#17214) · df6f3ce8
  Nick Hill authored Apr 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  df6f3ce8
- [CI/test] Fix Eagle Correctness Test (#17209) · 513f0747
  Woosuk Kwon authored Apr 25, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  513f0747
- [BugFix] Avoid race conditions in zero-copy tensor transmission (#17203) · b07bf83c
  Nick Hill authored Apr 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  b07bf83c
- [V1][Metrics] Allow V1 AsyncLLM to use custom logger (#14661) · 53e8cf53
  Zijing Liu authored Apr 25, 2025
```
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  53e8cf53
25 Apr, 2025 2 commits

[V1][Spec Decode] EAGLE-3 Support (#16937) · a0e619e6

Benjamin Chislett authored Apr 25, 2025


Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Co-authored-by: Bryan Lu <yuzhelu@amazon.com>

a0e619e6

[Bugfix] remove fallback in guided_json (int range, patterns) (#16725) · 6aae216b

Sangyeon Cho authored Apr 25, 2025


Signed-off-by: csy1204 <josang1204@gmail.com>
Co-authored-by: 조상연[플레이스 AI] <sang-yeon.cho@navercorp.com>

6aae216b

24 Apr, 2025 4 commits
- [V1][Spec Decoding] Add num_drafts and num_accepted_tokens_per_position metrics (#16665) · 340d7b1b
  Mark McLoughlin authored Apr 24, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  340d7b1b
- [V1][PP] Optimization: continue scheduling prefill chunks (#17080) · c0dfd975
  Rui Qiao authored Apr 24, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  c0dfd975
- Simplify `TokenizerGroup` (#16790) · 0a05ed57
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0a05ed57
- Disable enforce_eager for V1 TPU sampler and structured output tests (#17016) · 14288d13
  Michael Goin authored Apr 24, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  14288d13
23 Apr, 2025 3 commits
- [Frontend] Support guidance:no-additional-properties for compatibility with xgrammar (#15949) · 3cde34a4
  Travis Johnson authored Apr 23, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  3cde34a4
- [V1][DP] More robust DP/EP dummy request coordination (#16277) · 1e013fa3
  Nick Hill authored Apr 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  1e013fa3
- [Core][V1][TPU] Enable structured decoding on TPU V1 (#16499) · 83d93371
  Chenyaaang authored Apr 22, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  83d93371
22 Apr, 2025 4 commits

[V1] Remove pre-allocation for KV cache (#16941) · c4ab9f3e
Woosuk Kwon authored Apr 22, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
c4ab9f3e

[Bugfix] Fix the issue where llm.generate cannot be called repeatedly after... · acba33a0

Chauncey authored Apr 22, 2025


[Bugfix] Fix the issue where llm.generate cannot be called repeatedly after setting GuidedDecodingParams (#16767)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>

acba33a0

[Bugfix]: fix issue with n>1 sampling on v1 requests overriding each other (#16863) · 0e425449
Jeffrey Li authored Apr 21, 2025
```
Signed-off-by: Jeffrey Li <jeffrey.dot.li@gmail.com>
```
0e425449

[TPU][V1] Enable Top-P (#16843) · fa3bba2a

Nicolò Lucchesi authored Apr 22, 2025


Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>

fa3bba2a