Commits · 1c2bc7ead019cdf5b04b2f1d07b00982352f85ef · OpenDAS / vllm_cscc

30 Apr, 2025 1 commit

Truncation control for embedding models (#14776) · 1c2bc7ea

Gabriel Marinho authored Apr 29, 2025


Signed-off-by: Gabriel Marinho <gmarinho@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>

1c2bc7ea

29 Apr, 2025 2 commits
- Simplify (and fix) passing of guided decoding backend options (#17008) · a6977dbd
  Harry Mellor authored Apr 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a6977dbd
- [Misc] Move config fields to MultiModalConfig (#17343) · ebb3930d
  Cyrus Leung authored Apr 29, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ebb3930d
26 Apr, 2025 3 commits

[Core] Remove prompt string from engine core data structures (#17214) · df6f3ce8
Nick Hill authored Apr 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
df6f3ce8
[BugFix] Avoid race conditions in zero-copy tensor transmission (#17203) · b07bf83c
Nick Hill authored Apr 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
b07bf83c

[V1][Metrics] Allow V1 AsyncLLM to use custom logger (#14661) · 53e8cf53

Zijing Liu authored Apr 25, 2025


Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

53e8cf53

25 Apr, 2025 2 commits
- [V1] Move usage stats to worker and start logging TPU hardware (#16211) · 48cb2109
  Daniel Li authored Apr 25, 2025
  
  48cb2109
- Only turn on FastIncrementalDetokenizer when tokenizers >= 0.21.1 (#17158) · fc966e9c
  Lu Fang authored Apr 25, 2025
  
  fc966e9c
24 Apr, 2025 5 commits
- Add collective_rpc to llm engine (#16999) · fe921763
  Yinghai Lu authored Apr 24, 2025
```
Signed-off-by: Yinghai Lu <yinghai@thinkingmachines.ai>
```
  fe921763
- [V1][Structured Output] Clear xgrammar compiler object when engine core shut... · b724afe3
  Shanshan Shen authored Apr 24, 2025
```
[V1][Structured Output] Clear xgrammar compiler object when engine core shut down to avoid nanobind leaked warning (#16954)
Signed-off-by: shen-shanshan <467638484@qq.com>
```
  b724afe3
- [V1][PP] Optimization: continue scheduling prefill chunks (#17080) · c0dfd975
  Rui Qiao authored Apr 24, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  c0dfd975
- Simplify `TokenizerGroup` (#16790) · 0a05ed57
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0a05ed57
- [Bugfix] Enable V1 usage stats (#16986) · ed50f466
  Michael Goin authored Apr 23, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  ed50f466
23 Apr, 2025 4 commits
- [Frontend] Support guidance:no-additional-properties for compatibility with xgrammar (#15949) · 3cde34a4
  Travis Johnson authored Apr 23, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  3cde34a4
- Use `@property` and private field for `data_parallel_rank_local` (#17053) · bdb36603
  Harry Mellor authored Apr 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  bdb36603
- [V1] Avoid socket errors during shutdown when requests are in in-flight (#16807) · b2f195c4
  Nick Hill authored Apr 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  b2f195c4
- [V1][DP] More robust DP/EP dummy request coordination (#16277) · 1e013fa3
  Nick Hill authored Apr 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  1e013fa3
22 Apr, 2025 3 commits

[BugFix] Fix incremental detokenization perf issue (#16963) · e4d61442
Nick Hill authored Apr 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
e4d61442

[Bugfix] Fix the issue where llm.generate cannot be called repeatedly after... · acba33a0

Chauncey authored Apr 22, 2025


[Bugfix] Fix the issue where llm.generate cannot be called repeatedly after setting GuidedDecodingParams (#16767)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>

acba33a0

[Bugfix]: fix issue with n>1 sampling on v1 requests overriding each other (#16863) · 0e425449
Jeffrey Li authored Apr 21, 2025
```
Signed-off-by: Jeffrey Li <jeffrey.dot.li@gmail.com>
```
0e425449

18 Apr, 2025 1 commit
- [V1][Structured Output] Minor modification to `_validate_structured_output()` (#16748) · 30ed81b7
  Shanshan Shen authored Apr 18, 2025
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
  30ed81b7
17 Apr, 2025 5 commits

[P/D][V1] KV Connector API V1 (#15960) · 3408e471

Yihua Cheng authored Apr 17, 2025


Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: remi <remi@mistral.ai>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Rémi Delacourt <54138269+Flechman@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>

3408e471

[V1][Perf] Faster incremental detokenization (#15137) · 05fcd1b4
Nick Hill authored Apr 17, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
05fcd1b4
[Bugfix] Revert max_prompt_len validation for decoder-only models. (#16741) · 607029e5
David Heineman authored Apr 16, 2025
```
Signed-off-by: David Heineman <david@davidheineman.com>
```
607029e5

[V1][Frontend] Improve Shutdown And Logs (#11737) · 2b05b8ce

Robert Shaw authored Apr 16, 2025


Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

2b05b8ce

Improve error for structured output backend selection (#16717) · 93e561ec
Harry Mellor authored Apr 17, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
93e561ec

16 Apr, 2025 2 commits
- [Hardware] Add processor inputs to platform validation (#16680) · e1b00483
  Joe Runde authored Apr 16, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  e1b00483
- [V1][Structured Output] Move xgrammar related utils to `backend_xgrammar.py` (#16578) · 976711d9
  Shanshan Shen authored Apr 16, 2025
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
  976711d9
15 Apr, 2025 1 commit
- Add "/server_info" endpoint in api_server to retrieve the vllm_config. (#16572) · 1666e664
  Xihui Cang authored Apr 15, 2025
```
Signed-off-by: Xihui Cang <xihuicang@gmail.com>
```
  1666e664
14 Apr, 2025 1 commit
- [Bugfix] Multi-modal caches not acting like LRU caches (#16593) · aa29841e
  Cyrus Leung authored Apr 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  aa29841e
13 Apr, 2025 1 commit
- [V1][Spec Decode] KV cache slots for eagle heads (#16370) · f49e5aff
  Lily Liu authored Apr 12, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  f49e5aff
12 Apr, 2025 1 commit
- [Bugfix] Validate logit biases to prevent out of vocab ids crashing engine (#16529) · 6c11ecf8
  Ryan McConville authored Apr 12, 2025
```
Signed-off-by: Ryan McConville <ryan@ryanmcconville.com>
```
  6c11ecf8
10 Apr, 2025 2 commits
- [V1] Zero-copy tensor/ndarray serialization/transmission (#13790) · dd143ef5
  Nick Hill authored Apr 10, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  dd143ef5
- [Bugfix] Fix validation error for text-only Mllama 3.2 (#16377) · a5d11a54
  Cyrus Leung authored Apr 10, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  a5d11a54
09 Apr, 2025 2 commits
- [Hardware] add platform-specific request validation api (#16291) · cb391d85
  Joe Runde authored Apr 09, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  cb391d85
- [Bugfix] Avoid transferring cached multi-modal items from P0 to P1 (#16273) · e484e028
  Cyrus Leung authored Apr 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e484e028
08 Apr, 2025 1 commit
- [Bugfix] Proper input validation for multi-modal encoder-decoder models (#16156) · 4ebc0b96
  Cyrus Leung authored Apr 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  4ebc0b96
07 Apr, 2025 2 commits
- [V1][BugFix] Exit properly if engine core fails during startup (#16137) · 7f6d47c1
  Nick Hill authored Apr 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  7f6d47c1
- [V1][Structured Output] Add `supports_structured_output()` method to Platform (#16148) · e9ba99f2
  Shanshan Shen authored Apr 07, 2025
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
  e9ba99f2
04 Apr, 2025 1 commit
- [V1] DP scale-out (1/N): Use zmq ROUTER/DEALER sockets for input queue (#15906) · 651cf0fe
  Nick Hill authored Apr 04, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  651cf0fe