Commits · 4959915089f1bcf011f082136464e48b76c7e3d9 · OpenDAS / vllm_cscc

17 Jun, 2025 1 commit
- [Mis] remove duplicate engine status checks (#19647) · 5a1c2e15
  CYJiang authored Jun 17, 2025
```
Signed-off-by: googs1025 <googs1025@gmail.com>
```
  5a1c2e15
16 Jun, 2025 2 commits
- [Bugfix] Fix TP inference for Flex attention backend (#19657) · 1173804d
  Isotr0py authored Jun 16, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  1173804d
- [BugFix] Don't catch BaseException when dumping execute_model errors (#19626) · ee35e96a
  Nick Hill authored Jun 16, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  ee35e96a
14 Jun, 2025 1 commit
- [BugFix] Fix DP Coordinator incorrect debug log message (#19624) · bd517eb9
  Nick Hill authored Jun 13, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  bd517eb9
12 Jun, 2025 1 commit
- [BugFix] Work-around incremental detokenization edge case error (#19449) · d5bdf899
  Nick Hill authored Jun 11, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  d5bdf899
06 Jun, 2025 2 commits
- [Bugfix][Core] Update cancellation logic in `generate()` to handle Generator exits (#19225) · ca27f0f9
  Adolfo Victoria authored Jun 06, 2025
```
Co-authored-by: Adolfo Victoria <adovi@meta.com>
```
  ca27f0f9
- [Core] Raise when non-multi-instance DP clients target a DP rank (#19227) · 7353492a
  jmswen authored Jun 06, 2025
```
Signed-off-by: Jon Swenson <jmswen@gmail.com>
```
  7353492a
04 Jun, 2025 2 commits
- [Misc] refactor: simplify EngineCoreClient.make_async_mp_client in AsyncLLM (#18817) · 23027e2d
  CYJiang authored Jun 05, 2025
```
Signed-off-by: googs1025 <googs1025@gmail.com>
```
  23027e2d
- Allow AsyncLLMEngine.generate to target a specific DP rank (#19102) · c8dcc159
  jmswen authored Jun 04, 2025
```
Signed-off-by: Jon Swenson <jmswen@gmail.com>
```
  c8dcc159
03 Jun, 2025 3 commits
- [Misc] Add SPDX-FileCopyrightText (#19100) · 02f0c7b2
  Simon Mo authored Jun 03, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  02f0c7b2
- [Bugfix] disable processor cache (#19068) · d81edded
  Raushan Turganbay authored Jun 03, 2025
```
Signed-off-by: raushan <raushan@huggingface.co>
```
  d81edded
- [V1] Support DP with Ray (#18779) · bdce64f2
  Rui Qiao authored Jun 02, 2025
  
  bdce64f2
31 May, 2025 1 commit

[BugFix] Fix multi-node offline data-parallel (#18981) · 9a1b9b99

Nick Hill authored May 31, 2025


Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Yizhou Liu <liu_yizhou@outlook.com>

9a1b9b99

30 May, 2025 1 commit
- [Perf] API-server scaleout with many-to-many server-engine comms (#17546) · 2dbe8c07
  Nick Hill authored May 30, 2025
  
  2dbe8c07
29 May, 2025 3 commits
- [P/D] NixlConnector DP fixes (#18903) · 64eaf5fe
  Will Eaton authored May 29, 2025
```
Signed-off-by: Will Eaton <weaton@redhat.com>
```
  64eaf5fe
- [BugFix] Make DP work with connector-delayed new requests (#18559) · d1d61f33
  Nick Hill authored May 29, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Will Eaton <weaton@redhat.com>
```
  d1d61f33
- [Misc] Remove duplicate init for self.vllm_config (#18896) · d58f9c7f
  CYJiang authored May 30, 2025
```
Signed-off-by: googs1025 <googs1025@gmail.com>
```
  d58f9c7f
27 May, 2025 1 commit
- [V1][Metrics] Add API for accessing in-memory Prometheus metrics (#17010) · 06a03380
  Mark McLoughlin authored May 27, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  06a03380
25 May, 2025 1 commit

[V1] Fix _pickle.PicklingError: Can't pickle <class... · 7891fdf0

Seiji Eicher authored May 24, 2025


[V1] Fix _pickle.PicklingError: Can't pickle <class 'transformers_modules.deepseek-ai.DeepSeek-V2-Lite... (#18640)
Signed-off-by: Seiji Eicher <seiji@anyscale.com>

7891fdf0

23 May, 2025 2 commits
- [Feature][V1]: suupports cached_tokens in response usage (#18149) · b046cf79
  Chauncey authored May 23, 2025
```
Co-authored-by: simon-mo <xmo@berkeley.edu>
```
  b046cf79
- [Frontend] [Core] Add Tensorizer support for V1, LoRA adapter serialization... · c32e249a
  Sanger Steel authored May 22, 2025
```
[Frontend] [Core] Add Tensorizer support for V1, LoRA adapter serialization and deserialization (#17926)
Signed-off-by: Sanger Steel <sangersteel@gmail.com>
```
  c32e249a
22 May, 2025 1 commit
- [BugFix][DP] Send DP wave completion only from `dp_rank==0` (#18502) · cf5984b2
  Nick Hill authored May 21, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: kourosh hakhamaneshi <kourosh@anyscale.com>
```
  cf5984b2
21 May, 2025 1 commit
- [V1] Fix general plugins not loaded in engine for multiproc (#18326) · 907f935d
  Yong Hoon Shin authored May 21, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  907f935d
14 May, 2025 1 commit
- [KVConnector] Keep KVTransferParams as a dict (#18033) · 59dd311c
  Nick Hill authored May 14, 2025
  
  59dd311c
13 May, 2025 3 commits
- [V1] DP scale-out (2/N): Decouple engine process management and comms (#15977) · 55aa7af9
  Nick Hill authored May 13, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  55aa7af9
- [Bugfix] Fix entrypoints metrics tests (#18063) · b922c2eb
  Cyrus Leung authored May 13, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b922c2eb
- [Bugfix] Avoid repeatedly creating dummy data during engine startup (#17935) · 61e0a506
  Cyrus Leung authored May 13, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  61e0a506
12 May, 2025 3 commits

[Core] Use platform-agnostic device control for DP engine core (#17245) · 289199fe
Jade Zheng authored May 13, 2025
```
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
```
289199fe

[P/D] NIXL Integration (#17751) · d1911020

Robert Shaw authored May 12, 2025


Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Brent Salisbury <bsalisbu@redhat.com>

d1911020

[Bugfix] validate grammar and throw 400 error instead of crashing the engine... · 08bf7840

Cheng Kuan Yong Jason authored May 12, 2025


[Bugfix] validate grammar and throw 400 error instead of crashing the engine when xgrammar validation fails (#17623)
Signed-off-by: Jason Cheng <jasoncky96@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>

08bf7840

11 May, 2025 1 commit
- [Bugfix]: v1 engine - consider lora adapters in allowed_token_ids (#17855) · 8132365b
  Ben Browning authored May 11, 2025
```
Signed-off-by: Ben Browning <bbrownin@redhat.com>
```
  8132365b
08 May, 2025 1 commit
- [BugFix] Fix `--disable-log-stats` in V1 server mode (#17600) · 3d13ca0e
  Nick Hill authored May 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  3d13ca0e
07 May, 2025 1 commit
- [Core][Feature] Input metadata dump on crash (#13407) · d43f914d
  Wallas Henrique authored May 07, 2025
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  d43f914d
04 May, 2025 1 commit
- Add full API docs and improve the UX of navigating them (#17485) · d6484ef3
  Harry Mellor authored May 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6484ef3
30 Apr, 2025 4 commits

[V1][Metrics] add support for kv event publishing (#16750) · 0be6d05b

Alec authored Apr 30, 2025


Signed-off-by: alec-flowers <aflowers@nvidia.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>

0be6d05b

[Core] Prevent side-channel attacks via cache salting (#17045) · 77073c77
Marko Rosenmueller authored Apr 30, 2025
```
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
```
77073c77
[V1][Bugfix]: vllm v1 verison metric num_gpu_blocks is None (#15755) · d8037867
rongfu.leng authored Apr 30, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
d8037867

Truncation control for embedding models (#14776) · 1c2bc7ea

Gabriel Marinho authored Apr 29, 2025


Signed-off-by: Gabriel Marinho <gmarinho@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>

1c2bc7ea

29 Apr, 2025 2 commits
- Simplify (and fix) passing of guided decoding backend options (#17008) · a6977dbd
  Harry Mellor authored Apr 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a6977dbd
- [Misc] Move config fields to MultiModalConfig (#17343) · ebb3930d
  Cyrus Leung authored Apr 29, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ebb3930d