Commits · 2b0db9b0e2378dbb7c44dd17e4066b1f42d42b70 · OpenDAS / vllm_cscc

12 May, 2025 2 commits

[P/D] NIXL Integration (#17751) · d1911020

Robert Shaw authored May 12, 2025


Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Brent Salisbury <bsalisbu@redhat.com>

d1911020

[Bugfix] validate grammar and throw 400 error instead of crashing the engine... · 08bf7840

Cheng Kuan Yong Jason authored May 12, 2025


[Bugfix] validate grammar and throw 400 error instead of crashing the engine when xgrammar validation fails (#17623)
Signed-off-by: Jason Cheng <jasoncky96@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>

08bf7840

10 May, 2025 2 commits
- [v1] Rename specialized_manager.py to single_type_kv_cache_manager.py (#17946) · ca66a167
  Chen Zhang authored May 11, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  ca66a167
- [v1] Pass BlockTable and KVCacheSpec to AttentionMetadataBuilders (#17483) · 950751a9
  Chen Zhang authored May 11, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  950751a9
09 May, 2025 2 commits
- [v1] Move block management logic from KVCacheManager to SpecializedManager (#17474) · 200da9a5
  Chen Zhang authored May 09, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  200da9a5
- [BUGFIX]: return fast when request requires prompt logprobs (#17251) · d310e6de
  Ning Xie authored May 09, 2025
  
  d310e6de
08 May, 2025 2 commits
- [TPU] Fix the test_sampler (#17820) · a463555d
  Jevin Jiang authored May 08, 2025
  
  a463555d
- [V1] Add VLLM_ALLOW_INSECURE_SERIALIZATION env var (#17490) · 6930a411
  Russell Bryant authored May 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  6930a411
07 May, 2025 1 commit
- [Frontend] Add missing chat templates for various MLLMs (#17758) · 8a15c260
  Cyrus Leung authored May 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8a15c260
06 May, 2025 1 commit
- [v1] Introduce KVCacheBlocks as interface between Scheduler and KVCacheManager (#17479) · aabcd2ca
  Chen Zhang authored May 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  aabcd2ca
05 May, 2025 3 commits
- [TPU][V1] Add support for top-logprobs (#17072) · 5941e0b7
  Nicolò Lucchesi authored May 05, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  5941e0b7
- [TPU] Enable gemma3-27b with TP>1 on multi-chips. (#17335) · 97659408
  XiongfeiWei authored May 05, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  97659408
- [BugFix] Increase timeout for startup failure test (#17642) · 5ea5c514
  Nick Hill authored May 05, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  5ea5c514
04 May, 2025 1 commit
- Add full API docs and improve the UX of navigating them (#17485) · d6484ef3
  Harry Mellor authored May 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6484ef3
02 May, 2025 1 commit
- [BugFix] Fix Memory Leak (#17567) · c777df79
  Robert Shaw authored May 02, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  c777df79
01 May, 2025 3 commits
- [CI][Bugfix] Fix failing V1 Test due to missing 'cache_salt' arg (#17500) · aa4502e7
  Michael Goin authored Apr 30, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  aa4502e7
- [CI][TPU] Skip structured outputs+spec decode tests on TPU (#17510) · 17b4d85f
  Michael Goin authored Apr 30, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  17b4d85f
- [CI][TPU] Skip Multimodal test (#17488) · dbc18e78
  Siyuan Liu authored Apr 30, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  dbc18e78
30 Apr, 2025 6 commits
- [v1][Spec Decode] Make sliding window compatible with eagle prefix caching (#17398) · 81ecf425
  Chen Zhang authored May 01, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  81ecf425
- [V1] Allow turning off pickle fallback in vllm.v1.serial_utils (#17427) · 947f2f53
  Russell Bryant authored Apr 30, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  947f2f53
- [V1][Metrics] add support for kv event publishing (#16750) · 0be6d05b
  Alec authored Apr 30, 2025
```
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
```
  0be6d05b
- [Core] Prevent side-channel attacks via cache salting (#17045) · 77073c77
  Marko Rosenmueller authored Apr 30, 2025
```
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
```
  77073c77
- [TPU][V1][CI] Update regression test baseline for v6 CI (#17064) · a7d5b016
  Nicolò Lucchesi authored Apr 30, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  a7d5b016
- [V1][Feature] Enable Speculative Decoding with Structured Outputs (#14702) · 34120f5a
  Benjamin Chislett authored Apr 29, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
```
  34120f5a
29 Apr, 2025 4 commits
- Remove Bamba 9B from CI (#17407) · 7489ec0b
  Harry Mellor authored Apr 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  7489ec0b
- Simplify (and fix) passing of guided decoding backend options (#17008) · a6977dbd
  Harry Mellor authored Apr 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a6977dbd
- [Misc]Use a platform independent interface to obtain the device attributes (#17100) · bdb2cdda
  ponix-j authored Apr 29, 2025
  
  bdb2cdda
- implement Structural Tag with Guidance backend (#17333) · 86d9fc29
  Michał Moskal authored Apr 28, 2025
```
Signed-off-by: Michal Moskal <michal@moskal.me>
```
  86d9fc29
27 Apr, 2025 1 commit
- [V1][Spec Decode] Make eagle compatible with prefix caching. (#17137) · 20e489ea
  Lily Liu authored Apr 27, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  20e489ea
26 Apr, 2025 6 commits
- [MISC] rename interval to max_recent_requests (#14285) · fd11a325
  Ning Xie authored Apr 27, 2025
  
  fd11a325
- [V1] Add `structural_tag` support using xgrammar (#17085) · f8acd01f
  Russell Bryant authored Apr 26, 2025
  
  f8acd01f
- [Core] Remove prompt string from engine core data structures (#17214) · df6f3ce8
  Nick Hill authored Apr 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  df6f3ce8
- [CI/test] Fix Eagle Correctness Test (#17209) · 513f0747
  Woosuk Kwon authored Apr 25, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  513f0747
- [BugFix] Avoid race conditions in zero-copy tensor transmission (#17203) · b07bf83c
  Nick Hill authored Apr 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  b07bf83c
- [V1][Metrics] Allow V1 AsyncLLM to use custom logger (#14661) · 53e8cf53
  Zijing Liu authored Apr 25, 2025
```
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  53e8cf53
25 Apr, 2025 2 commits

[V1][Spec Decode] EAGLE-3 Support (#16937) · a0e619e6

Benjamin Chislett authored Apr 25, 2025


Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Co-authored-by: Bryan Lu <yuzhelu@amazon.com>

a0e619e6

[Bugfix] remove fallback in guided_json (int range, patterns) (#16725) · 6aae216b

Sangyeon Cho authored Apr 25, 2025


Signed-off-by: csy1204 <josang1204@gmail.com>
Co-authored-by: 조상연[플레이스 AI] <sang-yeon.cho@navercorp.com>

6aae216b

24 Apr, 2025 3 commits
- [V1][Spec Decoding] Add num_drafts and num_accepted_tokens_per_position metrics (#16665) · 340d7b1b
  Mark McLoughlin authored Apr 24, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  340d7b1b
- [V1][PP] Optimization: continue scheduling prefill chunks (#17080) · c0dfd975
  Rui Qiao authored Apr 24, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  c0dfd975
- Simplify `TokenizerGroup` (#16790) · 0a05ed57
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0a05ed57