Commits · ebcce2cd36a75effd10556942f0467f5f670a080 · OpenDAS / vllm_cscc

14 Aug, 2025 3 commits

[Core] Return final response for aborted requests from `AsyncLLM.generate` (#22283) · ebcce2cd
Nick Hill authored Aug 14, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
ebcce2cd

[Bugfix] Add reset prefix cache for online serving (#22726) · 7655dc3e

iAmir97 authored Aug 14, 2025

Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Signed-off-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

7655dc3e

[BugFix] Threadsafe close async zmq sockets (#22877) · eb08487b

Nick Hill authored Aug 14, 2025


Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

eb08487b

13 Aug, 2025 1 commit
- [Core] Use individual MM items in P0/P1 cache and model runner (#22570) · 19b927e5
  Cyrus Leung authored Aug 13, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  19b927e5
09 Aug, 2025 1 commit

Implicit language-model-only mode via limit-mm-per-prompt (#22299) · 08b751ba

Roger Wang authored Aug 08, 2025


Signed-off-by: Roger Wang <hey@rogerw.me>
Signed-off-by: Andy Xie <andy.xning@gmail.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: XIn Li <xinli@nvidia.com>
Signed-off-by: Junhao Li <junhao@ubicloud.com>
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
Signed-off-by: zitian zhao <zitian.zhao@tencentmusic.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Signed-off-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Signed-off-by: Linkun <github@lkchen.net>
Co-authored-by: Ning Xie <andy.xning@gmail.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Andrew Sansom <andrew@protopia.ai>
Co-authored-by: Zhiyu <zhiyuc@nvidia.com>
Co-authored-by: Shu Wang <shuw@nvidia.com>
Co-authored-by: XIn Li <xinli@nvidia.com>
Co-authored-by: Junhao Li <streaver91@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Yuxuan Zhang <2448370773@qq.com>
Co-authored-by: ZiTian Zhao <zitian.zhao@tencentmusic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Hong Hanh <hanh.usth@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: lkchen <github@lkchen.net>

08b751ba

08 Aug, 2025 2 commits
- [BugFix] Don't cancel asyncio tasks directly from destructors (#22476) · ccdae737
  Nick Hill authored Aug 08, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  ccdae737
- [CI/Build] Fix multimodal tests (#22491) · 1712543d
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1712543d
07 Aug, 2025 1 commit
- [Core] Store only the keys for multi-modal data in P0 (#22198) · 766bc816
  Cyrus Leung authored Aug 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  766bc816
06 Aug, 2025 1 commit
- [Bugfix] Skip dead and non-GPU nodes for Ray DP engine allocation (#22275) · 302962e8
  Rui Qiao authored Aug 05, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  302962e8
04 Aug, 2025 1 commit
- [Sampler] Support returning all logprobs or logits (#21792) · 54de71d0
  22quinn authored Aug 04, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  54de71d0
02 Aug, 2025 1 commit
- [BugFix] Improve internal DP load balancing (#21617) · 8d524ce7
  Nick Hill authored Aug 02, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  8d524ce7
01 Aug, 2025 4 commits
- [BugFix] Harden distributed DP startup (#21538) · 881e1af4
  Nick Hill authored Aug 01, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  881e1af4
- [Bugfix] fix when skip tokenizer init (#21922) · b879ecd6
  rongfu.leng authored Aug 02, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  b879ecd6
- Deprecate `--disable-log-requests` and replace with `--enable-log-requests` (#21739) · 2d7b09b9
  Harry Mellor authored Aug 01, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  2d7b09b9
- [Bugfix] Add log prefix in non-dp mode engine core (#21889) · e6680f9e
  wuhang authored Aug 01, 2025
```
Signed-off-by: wuhang <wuhang6@huawei.com>
```
  e6680f9e
30 Jul, 2025 3 commits
- [Core] Move EngineCoreRequest to Request conversion out of EngineCore (#21627) · ca9e2be3
  Zebing Lin authored Jul 30, 2025
```
Signed-off-by: linzebing <linzebing1995@gmail.com>
```
  ca9e2be3
- [Misc] Support more collective_rpc return types (#21845) · 56bd537d
  Nick Hill authored Jul 30, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  56bd537d
- [Bugfix] Fix TypeError in scheduler when comparing mixed request_id types (#21816) · 5c765aec
  633WHU authored Jul 30, 2025
```
Signed-off-by: chiliu <chiliu@paypal.com>
Co-authored-by: chiliu <chiliu@paypal.com>
```
  5c765aec
29 Jul, 2025 1 commit
- [Misc] Rework process titles (#21780) · 7234fe26
  Nick Hill authored Jul 29, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  7234fe26
28 Jul, 2025 1 commit
- [Bugfix]check health for engine core process exiting unexpectedly (#21728) · bccc43c0
  wuhang authored Jul 28, 2025
```
Signed-off-by: wuhang <wuhang6@huawei.com>
```
  bccc43c0
26 Jul, 2025 1 commit

Support encoder-only models without KV-Cache (#21270) · 1cd6eaba

Maximilien de Bayser authored Jul 26, 2025


Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>

1cd6eaba

25 Jul, 2025 1 commit
- [V1] Get supported tasks from model runner instead of model config (#21585) · 46d81d69
  Cyrus Leung authored Jul 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  46d81d69
24 Jul, 2025 5 commits

[Feat] Allow custom naming of vLLM processes (#21445) · 6da00785
Chauncey authored Jul 24, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
6da00785
[Misc] Improve comment for DPEngineCoreActor._set_cuda_visible_devices() (#21501) · 73e3949d
Rui Qiao authored Jul 24, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
73e3949d

[BugFix] Set CUDA_VISIBLE_DEVICES before spawning the subprocesses (#21211) · 11ef7a61

Yinghai Lu authored Jul 23, 2025


Signed-off-by: Yinghai Lu <yinghai@thinkingmachines.ai>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>

11ef7a61

Dump input metadata on crash for async scheduling (#21258) · dc2f159f
Woosuk Kwon authored Jul 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
dc2f159f

[DP] Internal Load Balancing Per Node [`one-pod-per-node`] (#21238) · d5b981f8

Robert Shaw authored Jul 23, 2025


Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

d5b981f8

23 Jul, 2025 2 commits
- [V0 Deprecation] Remove Prompt Adapters (#20588) · 82ec66f5
  Michael Goin authored Jul 23, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  82ec66f5
- [Core][Model] PrithviMAE Enablement on vLLM v1 engine (#20577) · 8560a5b2
  Christian Pinto authored Jul 23, 2025
```
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
```
  8560a5b2
22 Jul, 2025 1 commit
- Add tokenization_kwargs to encode for embedding model truncation (#21033) · 44554a00
  Wang Yijun authored Jul 22, 2025
  
  44554a00
21 Jul, 2025 1 commit

[DP] Fix Prometheus Logging (#21257) · 29d1ffc5

Robert Shaw authored Jul 21, 2025


Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>

29d1ffc5

19 Jul, 2025 1 commit
- Elastic Expert Parallel Initial Support (#20775) · 21793722
  Rui Qiao authored Jul 18, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  21793722
18 Jul, 2025 1 commit
- [Core] Set pooling params based on task and model (#21128) · 45badd05
  Cyrus Leung authored Jul 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  45badd05
15 Jul, 2025 1 commit
- [v1][core] Support for attention free models (#20811) · 4ffd963f
  Christian Pinto authored Jul 15, 2025
```
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
```
  4ffd963f
10 Jul, 2025 1 commit
- [V0][V1][Core] Add outlines integration for V1, and update V0 integration. (#15975) · d6902ce7
  Nathan Hoos authored Jul 10, 2025
```
Signed-off-by: Nathan Hoos <thwackyy.y@gmail.com>
```
  d6902ce7
07 Jul, 2025 1 commit
- [DP] Copy environment variables to Ray DPEngineCoreActors (#20344) · a6d795d5
  Rui Qiao authored Jul 07, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  a6d795d5
04 Jul, 2025 1 commit
- [Structured Outputs][V1] Skipping with models doesn't contain tokenizers (#20365) · 4a98edff
  Aaron Pham authored Jul 04, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  4a98edff
02 Jul, 2025 1 commit
- [DP] Support external DP Load Balancer mode (#19790) · 657f2f30
  Nick Hill authored Jul 02, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  657f2f30
28 Jun, 2025 2 commits
- [Bugfix] Properly reject requests with empty list guided_choice (#20195) · f7197722
  Michael Goin authored Jun 28, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  f7197722
- [Misc] Add type assertion of request_id for LLMEngine.add_request (#19700) · e53be6f0
  Chales Xu authored Jun 28, 2025
```
Signed-off-by: n2ptr <xuzhanchaomail@163.com>
```
  e53be6f0