Commits · 7a985548488ddbfa965f9684acf6f1d53f905964 · OpenDAS / vllm_cscc

19 May, 2025 1 commit
- [BugFix] Fix handling of num_computed_tokens with connector (#18232) · 1b15df25
  Nick Hill authored May 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
```
  1b15df25
17 May, 2025 3 commits

[Hardware][TPU] Optionally import for TPU backend (#18269) · 48ac2bed

Siyuan Liu authored May 17, 2025


Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
Co-authored-by: Carol Zheng <cazheng@google.com>
Co-authored-by: Jade Zheng <zheng.shoujian@outlook.com>
Co-authored-by: Hongmin Fan <fanhongmin@google.com>

48ac2bed

[P/D][V1] Support dynamic loading of external KV connector implementations (#18142) · 3e0d4350
David Ben-David authored May 17, 2025
```
Signed-off-by: David Ben-David <davidb@pliops.com>
Co-authored-by: David Ben-David <davidb@pliops.com>
```
3e0d4350
[V1][P/D] Local attention optimization for NIXL (#18170) · fd195b19
Michael Goin authored May 16, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
fd195b19

16 May, 2025 3 commits
- [BugFix] Fix ordering of KVConnector finished send/rcv sets (#18211) · 0ceaebf8
  Nick Hill authored May 16, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  0ceaebf8
- [BugFix] Fix multi async save in MultiConnector (#18246) · 1db4f47f
  Nick Hill authored May 16, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  1db4f47f
- [Feature] Support Pipeline Parallism in torchrun SPMD offline inference for V1 (#17827) · 3d2779c2
  Lucia Fang authored May 15, 2025
```
Signed-off-by: Lucia Fang <fanglu@fb.com>
```
  3d2779c2
15 May, 2025 2 commits
- [V1] Update zmq socket creation in nixl connector (#18148) · a8f5aec2
  Russell Bryant authored May 15, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  a8f5aec2
- [v1] Support multiple KV cache groups in GPU model runner (#17945) · e60f550b
  Chen Zhang authored May 15, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  e60f550b
14 May, 2025 7 commits
- [V1] Support multiple kv connectors (#17564) · 2142035b
  Michael Goin authored May 14, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  2142035b
- [CI] Fix race condition in test_kv_cache_events test (#18169) · 78aa341d
  Russell Bryant authored May 14, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  78aa341d
- Modularize fused experts and integrate PPLX kernels (#15956) · f9c069c8
  bnellnm authored May 14, 2025
  
  f9c069c8
- [KVConnector] Keep KVTransferParams as a dict (#18033) · 59dd311c
  Nick Hill authored May 14, 2025
  
  59dd311c
- Update deprecated type hinting in `vllm/device_allocator` and `vllm/distributed` (#18126) · dc372b9c
  Harry Mellor authored May 14, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  dc372b9c
- [core][distributed] add ep group and all2all interface (#18077) · 6266c57b
  youkaichao authored May 14, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  6266c57b
- [P/D] Add some more debug logs to `NixlConnector` (#18102) · d5af47a1
  Nick Hill authored May 13, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  d5af47a1
13 May, 2025 1 commit
- [V1] DP scale-out (2/N): Decouple engine process management and comms (#15977) · 55aa7af9
  Nick Hill authored May 13, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  55aa7af9
12 May, 2025 1 commit

[P/D] NIXL Integration (#17751) · d1911020

Robert Shaw authored May 12, 2025


Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Brent Salisbury <bsalisbu@redhat.com>

d1911020

09 May, 2025 2 commits
- Improve configs - the rest! (#17562) · 4b2ed792
  Harry Mellor authored May 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  4b2ed792
- [Bugfix][CPU] Fix broken AVX2 CPU TP support (#17252) · 6e4a93e3
  Isotr0py authored May 09, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  6e4a93e3
05 May, 2025 1 commit
- [Bugfix][Easy] Fix whitespace in shm_broadcast.py logging (#17635) · 68e1ee00
  Tyler Michael Smith authored May 04, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  68e1ee00
04 May, 2025 1 commit
- Add full API docs and improve the UX of navigating them (#17485) · d6484ef3
  Harry Mellor authored May 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6484ef3
30 Apr, 2025 1 commit

[V1][Metrics] add support for kv event publishing (#16750) · 0be6d05b

Alec authored Apr 30, 2025


Signed-off-by: alec-flowers <aflowers@nvidia.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>

0be6d05b

28 Apr, 2025 1 commit
- [Security] Don't bind tcp zmq socket to all interfaces (#17197) · a0304dc5
  Russell Bryant authored Apr 28, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  a0304dc5
27 Apr, 2025 1 commit

[Feature] support sequence parallelism using compilation pass (#16155) · 690fe019

cascade authored Apr 27, 2025


Signed-off-by: cascade812 <cascade812@outlook.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

690fe019

26 Apr, 2025 1 commit
- [v1] [P/D] Adding LMCache KV connector for v1 (#16625) · 5e83a727
  Yihua Cheng authored Apr 25, 2025
  
  5e83a727
25 Apr, 2025 2 commits
- [Security] Use safe serialization and fix zmq setup for mooncake pipe (#17192) · a5450f11
  Russell Bryant authored Apr 25, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  a5450f11
- Use custom address for listening socket (#15988) · 0d6e187e
  jglaser authored Apr 24, 2025
```
Signed-off-by: Jens Glaser <glaserj@ornl.gov>
```
  0d6e187e
24 Apr, 2025 1 commit
- Addendum Fix to support FIPS enabled machines with MD5 hashing (#17043) · ed2e4646
  Areeb Syed authored Apr 24, 2025
```
Signed-off-by: sydarb <areebsyed237@gmail.com>
```
  ed2e4646
17 Apr, 2025 2 commits

[P/D][V1] KV Connector API V1 (#15960) · 3408e471

Yihua Cheng authored Apr 17, 2025


Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: remi <remi@mistral.ai>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Rémi Delacourt <54138269+Flechman@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>

3408e471

[V1][Frontend] Improve Shutdown And Logs (#11737) · 2b05b8ce

Robert Shaw authored Apr 16, 2025


Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

2b05b8ce

16 Apr, 2025 1 commit
- [Feature] add model aware kv ops helper (#16020) · 3ac98edc
  billishyahao authored Apr 16, 2025
```
Signed-off-by: billishyahao <bill.he@amd.com>
```
  3ac98edc
14 Apr, 2025 1 commit
- [Misc] Update usage with mooncake lib for kv transfer (#16523) · 1dd23386
  shangmingc authored Apr 14, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  1dd23386
12 Apr, 2025 1 commit
- [MISC] Make GroupCoordinator compatible with out-of-tree devices (#16464) · 68bb122e
  Huazhong Ji authored Apr 12, 2025
```
Signed-off-by: hzji210@gmail.com <hzji210@gmail.com>
```
  68bb122e
08 Apr, 2025 2 commits
- [Misc] Merge the logs of pp layers partitions (#16225) · b4ac449a
  Kebe authored Apr 08, 2025
```
Signed-off-by: Kebe <mail@kebe7jun.com>
```
  b4ac449a
- update custom_all_reduce · bfc2d6f7
  zhuwenwen authored Apr 08, 2025
  
  bfc2d6f7
03 Apr, 2025 1 commit
- [TPU] optimize the all-reduce performance (#15903) · 01b61136
  Chengji Yao authored Apr 02, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  01b61136
02 Apr, 2025 1 commit
- [CPU][Bugfix] Using custom allreduce for CPU backend (#15934) · 550b2801
  Li, Jiang authored Apr 02, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  550b2801
01 Apr, 2025 1 commit
- [Distributed] Add custom allreduce support for ROCM (#14125) · b7b7676d
  Ilya Markov authored Apr 01, 2025
```
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
```
  b7b7676d
29 Mar, 2025 1 commit
- [Feature][Disaggregated] Support XpYd disaggregated prefill with MooncakeStore (#12957) · 6fa7cd3d
  shangmingc authored Mar 29, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  6fa7cd3d