Commits · 87d06573343e0b3e98096153c9cce1f058f8f716 · OpenDAS / vllm_cscc

03 Jun, 2025 1 commit
- [tests]skip tpu and weight_loading tests, fix tests of worker · 5e078c69
  zhuwenwen authored Jun 03, 2025
  
  5e078c69
02 May, 2025 1 commit
- [BugFix] Fix Memory Leak (#17567) · edb5286e
  Robert Shaw authored May 02, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  edb5286e
27 Apr, 2025 1 commit
- [V1][Spec Decode] Make eagle compatible with prefix caching. (#17137) · 20e489ea
  Lily Liu authored Apr 27, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  20e489ea
26 Apr, 2025 6 commits
- [MISC] rename interval to max_recent_requests (#14285) · fd11a325
  Ning Xie authored Apr 27, 2025
  
  fd11a325
- [V1] Add `structural_tag` support using xgrammar (#17085) · f8acd01f
  Russell Bryant authored Apr 26, 2025
  
  f8acd01f
- [Core] Remove prompt string from engine core data structures (#17214) · df6f3ce8
  Nick Hill authored Apr 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  df6f3ce8
- [CI/test] Fix Eagle Correctness Test (#17209) · 513f0747
  Woosuk Kwon authored Apr 25, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  513f0747
- [BugFix] Avoid race conditions in zero-copy tensor transmission (#17203) · b07bf83c
  Nick Hill authored Apr 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  b07bf83c
- [V1][Metrics] Allow V1 AsyncLLM to use custom logger (#14661) · 53e8cf53
  Zijing Liu authored Apr 25, 2025
```
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  53e8cf53
25 Apr, 2025 2 commits

[V1][Spec Decode] EAGLE-3 Support (#16937) · a0e619e6

Benjamin Chislett authored Apr 25, 2025


Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Co-authored-by: Bryan Lu <yuzhelu@amazon.com>

a0e619e6

[Bugfix] remove fallback in guided_json (int range, patterns) (#16725) · 6aae216b

Sangyeon Cho authored Apr 25, 2025


Signed-off-by: csy1204 <josang1204@gmail.com>
Co-authored-by: 조상연[플레이스 AI] <sang-yeon.cho@navercorp.com>

6aae216b

24 Apr, 2025 4 commits
- [V1][Spec Decoding] Add num_drafts and num_accepted_tokens_per_position metrics (#16665) · 340d7b1b
  Mark McLoughlin authored Apr 24, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  340d7b1b
- [V1][PP] Optimization: continue scheduling prefill chunks (#17080) · c0dfd975
  Rui Qiao authored Apr 24, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  c0dfd975
- Simplify `TokenizerGroup` (#16790) · 0a05ed57
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0a05ed57
- Disable enforce_eager for V1 TPU sampler and structured output tests (#17016) · 14288d13
  Michael Goin authored Apr 24, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  14288d13
23 Apr, 2025 3 commits
- [Frontend] Support guidance:no-additional-properties for compatibility with xgrammar (#15949) · 3cde34a4
  Travis Johnson authored Apr 23, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  3cde34a4
- [V1][DP] More robust DP/EP dummy request coordination (#16277) · 1e013fa3
  Nick Hill authored Apr 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  1e013fa3
- [Core][V1][TPU] Enable structured decoding on TPU V1 (#16499) · 83d93371
  Chenyaaang authored Apr 22, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  83d93371
22 Apr, 2025 6 commits

[V1] Remove pre-allocation for KV cache (#16941) · c4ab9f3e
Woosuk Kwon authored Apr 22, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
c4ab9f3e

[Bugfix] Fix the issue where llm.generate cannot be called repeatedly after... · acba33a0

Chauncey authored Apr 22, 2025


[Bugfix] Fix the issue where llm.generate cannot be called repeatedly after setting GuidedDecodingParams (#16767)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>

acba33a0

[Bugfix]: fix issue with n>1 sampling on v1 requests overriding each other (#16863) · 0e425449
Jeffrey Li authored Apr 21, 2025
```
Signed-off-by: Jeffrey Li <jeffrey.dot.li@gmail.com>
```
0e425449

[TPU][V1] Enable Top-P (#16843) · fa3bba2a

Nicolò Lucchesi authored Apr 22, 2025


Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>

fa3bba2a

[V1] V1 FlashInfer Attention (#16684) · 986537f1

Michael Goin authored Apr 21, 2025


Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Aurick Qiao <qiao@aurick.net>

986537f1

[TPU][V1] Capture multimodal encoder during model compilation (#15051) · 21020752

Nicolò Lucchesi authored Apr 22, 2025


Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Siyuan Liu <lsiyuan@google.com>

21020752

21 Apr, 2025 3 commits
- [TPU][V1] Implicitly adjust page size when there's SMEM OOM (#16871) · 471fe656
  Chengji Yao authored Apr 21, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  471fe656
- [V1][Spec Decode] Handle draft tokens beyond max_model_len (#16087) · 3a0fba5c
  Woosuk Kwon authored Apr 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3a0fba5c
- [Bugfix] Fix v1/spec_decode/test_ngram.py (#16895) · bb3605db
  qizixi authored Apr 20, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  bb3605db
19 Apr, 2025 2 commits
- Serialize tensors using int8 views (#16866) · 87aaadef
  Staszek Paśko authored Apr 19, 2025
```
Signed-off-by: Staszek Pasko <staszek@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  87aaadef
- [V1][Misc] stop update prefix cache stats when logs_stats is disabled (#16460) · d9737ca1
  vie-serendipity authored Apr 19, 2025
```
Signed-off-by: vie-serendipity <2733147505@qq.com>
```
  d9737ca1
17 Apr, 2025 5 commits

[P/D][V1] KV Connector API V1 (#15960) · 3408e471

Yihua Cheng authored Apr 17, 2025


Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: remi <remi@mistral.ai>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Rémi Delacourt <54138269+Flechman@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>

3408e471

[V1][TPU] Enable Top K (#15489) · eb5819b2

Nicolò Lucchesi authored Apr 17, 2025


Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
Co-authored-by: Hyesoo Yang <hyeygit@gmail.com>

eb5819b2

[TPU][V1] Fix padding recompilation when `max-num-batched-tokens` is not even (#16726) · 5989f468
Nicolò Lucchesi authored Apr 17, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
5989f468

[V1][Frontend] Improve Shutdown And Logs (#11737) · 2b05b8ce

Robert Shaw authored Apr 16, 2025


Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

2b05b8ce

[V1][Performance] Implement custom serializaton for MultiModalKwargs [Rebased] (#16432) · 3092375e

Staszek Paśko authored Apr 17, 2025


Signed-off-by: Staszek Pasko <staszek@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

3092375e

16 Apr, 2025 1 commit
- [V1][Structured Output] Move xgrammar related utils to `backend_xgrammar.py` (#16578) · 976711d9
  Shanshan Shen authored Apr 16, 2025
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
  976711d9
14 Apr, 2025 1 commit
- [TPU][V1] Fix exponential padding when `max-num-batched-tokens` is not a power of 2 (#16596) · b3f2fddd
  Nicolò Lucchesi authored Apr 14, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  b3f2fddd
13 Apr, 2025 1 commit
- [V1][Spec Decode] KV cache slots for eagle heads (#16370) · f49e5aff
  Lily Liu authored Apr 12, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  f49e5aff
12 Apr, 2025 2 commits
- [Feature][V1] Add xgrammar to support minLength, maxLength with test (#16516) · e92d7085
  leon-seidel authored Apr 12, 2025
```
Signed-off-by: Leon Seidel <leon.seidel@fau.de>
```
  e92d7085
- [BugFix] Handle non-contiguous tensors properly when serializing (#16492) · 41cc883c
  Nick Hill authored Apr 11, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  41cc883c
11 Apr, 2025 1 commit
- Enforce valid max_num_batched_tokens when disable_chunked_mm_input=True (#16447) · aa3b3d76
  Michael Goin authored Apr 11, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  aa3b3d76