Commits · 217937221b6845913502371aba554a3357fbccfb · OpenDAS / vllm_cscc

19 Jul, 2025 1 commit
- Elastic Expert Parallel Initial Support (#20775) · 21793722
  Rui Qiao authored Jul 18, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  21793722
19 Jun, 2025 2 commits

[Bugfix] Add check_health to v1 async client. (#19821) · e2148dc5
kourosh hakhamaneshi authored Jun 18, 2025
```
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
```
e2148dc5

Support embedding models in V1 (#16188) · 799397ee

Maximilien de Bayser authored Jun 19, 2025


Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>

799397ee

06 Jun, 2025 1 commit
- [Bugfix][Core] Update cancellation logic in `generate()` to handle Generator exits (#19225) · ca27f0f9
  Adolfo Victoria authored Jun 06, 2025
```
Co-authored-by: Adolfo Victoria <adovi@meta.com>
```
  ca27f0f9
04 Jun, 2025 2 commits
- [Misc] refactor: simplify EngineCoreClient.make_async_mp_client in AsyncLLM (#18817) · 23027e2d
  CYJiang authored Jun 05, 2025
```
Signed-off-by: googs1025 <googs1025@gmail.com>
```
  23027e2d
- Allow AsyncLLMEngine.generate to target a specific DP rank (#19102) · c8dcc159
  jmswen authored Jun 04, 2025
```
Signed-off-by: Jon Swenson <jmswen@gmail.com>
```
  c8dcc159
03 Jun, 2025 2 commits
- [Misc] Add SPDX-FileCopyrightText (#19100) · 02f0c7b2
  Simon Mo authored Jun 03, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  02f0c7b2
- [V1] Support DP with Ray (#18779) · bdce64f2
  Rui Qiao authored Jun 02, 2025
  
  bdce64f2
30 May, 2025 1 commit
- [Perf] API-server scaleout with many-to-many server-engine comms (#17546) · 2dbe8c07
  Nick Hill authored May 30, 2025
  
  2dbe8c07
25 May, 2025 1 commit

[V1] Fix _pickle.PicklingError: Can't pickle <class... · 7891fdf0

Seiji Eicher authored May 24, 2025


[V1] Fix _pickle.PicklingError: Can't pickle <class 'transformers_modules.deepseek-ai.DeepSeek-V2-Lite... (#18640)
Signed-off-by: Seiji Eicher <seiji@anyscale.com>

7891fdf0

13 May, 2025 1 commit
- [Bugfix] Avoid repeatedly creating dummy data during engine startup (#17935) · 61e0a506
  Cyrus Leung authored May 13, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  61e0a506
08 May, 2025 1 commit
- [BugFix] Fix `--disable-log-stats` in V1 server mode (#17600) · 3d13ca0e
  Nick Hill authored May 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  3d13ca0e
30 Apr, 2025 2 commits
- [V1][Bugfix]: vllm v1 verison metric num_gpu_blocks is None (#15755) · d8037867
  rongfu.leng authored Apr 30, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  d8037867
- Truncation control for embedding models (#14776) · 1c2bc7ea
  Gabriel Marinho authored Apr 29, 2025
```
Signed-off-by: Gabriel Marinho <gmarinho@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
```
  1c2bc7ea
26 Apr, 2025 2 commits

[Core] Remove prompt string from engine core data structures (#17214) · df6f3ce8
Nick Hill authored Apr 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
df6f3ce8

[V1][Metrics] Allow V1 AsyncLLM to use custom logger (#14661) · 53e8cf53

Zijing Liu authored Apr 25, 2025


Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

53e8cf53

25 Apr, 2025 1 commit
- [V1] Move usage stats to worker and start logging TPU hardware (#16211) · 48cb2109
  Daniel Li authored Apr 25, 2025
  
  48cb2109
24 Apr, 2025 3 commits

Add collective_rpc to llm engine (#16999) · fe921763
Yinghai Lu authored Apr 24, 2025
```
Signed-off-by: Yinghai Lu <yinghai@thinkingmachines.ai>
```
fe921763
Simplify `TokenizerGroup` (#16790) · 0a05ed57
Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
0a05ed57

[Bugfix] Enable V1 usage stats (#16986) · ed50f466

Michael Goin authored Apr 23, 2025


Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

ed50f466

17 Apr, 2025 1 commit

[V1][Frontend] Improve Shutdown And Logs (#11737) · 2b05b8ce

Robert Shaw authored Apr 16, 2025


Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

2b05b8ce

15 Apr, 2025 1 commit
- Add "/server_info" endpoint in api_server to retrieve the vllm_config. (#16572) · 1666e664
  Xihui Cang authored Apr 15, 2025
```
Signed-off-by: Xihui Cang <xihuicang@gmail.com>
```
  1666e664
02 Apr, 2025 1 commit
- [core] Add tags parameter to wake_up() (#15500) · ddb94c26
  Eric Tang authored Apr 02, 2025
```
Signed-off-by: Eric <erictang000@gmail.com>
```
  ddb94c26
28 Mar, 2025 1 commit
- [V1] Remove legacy input registry (#15673) · 355f6634
  Cyrus Leung authored Mar 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  355f6634
27 Mar, 2025 1 commit
- [V1] AsyncLLM data parallel (#13923) · 15dac210
  Nick Hill authored Mar 27, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  15dac210
24 Mar, 2025 1 commit

[V1][Perf] Simpler request output queues (#15156) · 9d72daf4

Nick Hill authored Mar 24, 2025


Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>

9d72daf4

21 Mar, 2025 1 commit
- [V1] Avoid redundant input processing in n>1 case (#14985) · da6ea29f
  Nick Hill authored Mar 20, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  da6ea29f
19 Mar, 2025 1 commit
- [FEAT]Support reset prefix cache by specified device (#15003) · 26dd972a
  maobaolong authored Mar 20, 2025
  
  26dd972a
15 Mar, 2025 2 commits

[Core] Expose API endpoint `/is_sleeping` (#14312) · 74bc397b
Jun Duan authored Mar 15, 2025
```
Signed-off-by: Jun Duan <jun.duan.phd@outlook.com>
```
74bc397b

[V1] V1 Enablement Oracle (#13726) · d4d93db2

Robert Shaw authored Mar 15, 2025


Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

d4d93db2

12 Mar, 2025 1 commit
- [BugFix][V1] Fix parallel sampling finishing/aborts (#14512) · f5d3acd4
  Nick Hill authored Mar 12, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  f5d3acd4
07 Mar, 2025 2 commits

[V1] Eagerly remove finished requests from the batch (#14388) · 8ed5421a
Nick Hill authored Mar 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
8ed5421a

[V1][Core] Support for Structured Outputs (#12388) · 80e9afb5

Aaron Pham authored Mar 07, 2025


Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

80e9afb5

03 Mar, 2025 3 commits
- [V1] Simplify stats logging (#14082) · 872db2be
  Nick Hill authored Mar 03, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  872db2be
- [V1] Refactor parallel sampling support (#13774) · 4167252e
  Mark McLoughlin authored Mar 03, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  4167252e
- Update deprecated Python 3.8 typing (#13971) · cf069aa8
  Harry Mellor authored Mar 03, 2025
  
  cf069aa8
25 Feb, 2025 1 commit
- [Core] LoRA V1 - Add add/pin/list/remove_lora functions (#13705) · 03f48b3d
  Varun Sundar Rabindranath authored Feb 25, 2025
  
  03f48b3d
24 Feb, 2025 1 commit
- [V1] V1 engine implements parallel sampling (AsyncLLM and LLMEngine) (#10980) · befc402d
  afeldman-nm authored Feb 24, 2025
```
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  befc402d
20 Feb, 2025 1 commit

[core] add sleep and wake up endpoint and v1 support (#12987) · ba811639

youkaichao authored Feb 20, 2025


Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: cennn <2523403608@qq.com>
Co-authored-by: cennn <2523403608@qq.com>

ba811639

15 Feb, 2025 1 commit
- [V1][Metrics] Add iteration_tokens_total histogram from V0 (#13288) · 2ad1bc7a
  Mark McLoughlin authored Feb 15, 2025
  
  2ad1bc7a