Commits · 0b217da646fd4cc08cd0dd20d0ea69f81d64ab35 · OpenDAS / vllm_cscc

12 May, 2025 1 commit

[P/D] NIXL Integration (#17751) · d1911020

Robert Shaw authored May 12, 2025


Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Brent Salisbury <bsalisbu@redhat.com>

d1911020

09 May, 2025 2 commits
- Improve configs - the rest! (#17562) · 4b2ed792
  Harry Mellor authored May 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  4b2ed792
- [Bugfix][CPU] Fix broken AVX2 CPU TP support (#17252) · 6e4a93e3
  Isotr0py authored May 09, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  6e4a93e3
05 May, 2025 1 commit
- [Bugfix][Easy] Fix whitespace in shm_broadcast.py logging (#17635) · 68e1ee00
  Tyler Michael Smith authored May 04, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  68e1ee00
04 May, 2025 1 commit
- Add full API docs and improve the UX of navigating them (#17485) · d6484ef3
  Harry Mellor authored May 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6484ef3
30 Apr, 2025 1 commit

[V1][Metrics] add support for kv event publishing (#16750) · 0be6d05b

Alec authored Apr 30, 2025


Signed-off-by: alec-flowers <aflowers@nvidia.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>

0be6d05b

28 Apr, 2025 1 commit
- [Security] Don't bind tcp zmq socket to all interfaces (#17197) · a0304dc5
  Russell Bryant authored Apr 28, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  a0304dc5
27 Apr, 2025 1 commit

[Feature] support sequence parallelism using compilation pass (#16155) · 690fe019

cascade authored Apr 27, 2025


Signed-off-by: cascade812 <cascade812@outlook.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

690fe019

26 Apr, 2025 1 commit
- [v1] [P/D] Adding LMCache KV connector for v1 (#16625) · 5e83a727
  Yihua Cheng authored Apr 25, 2025
  
  5e83a727
25 Apr, 2025 2 commits
- [Security] Use safe serialization and fix zmq setup for mooncake pipe (#17192) · a5450f11
  Russell Bryant authored Apr 25, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  a5450f11
- Use custom address for listening socket (#15988) · 0d6e187e
  jglaser authored Apr 24, 2025
```
Signed-off-by: Jens Glaser <glaserj@ornl.gov>
```
  0d6e187e
24 Apr, 2025 1 commit
- Addendum Fix to support FIPS enabled machines with MD5 hashing (#17043) · ed2e4646
  Areeb Syed authored Apr 24, 2025
```
Signed-off-by: sydarb <areebsyed237@gmail.com>
```
  ed2e4646
17 Apr, 2025 2 commits

[P/D][V1] KV Connector API V1 (#15960) · 3408e471

Yihua Cheng authored Apr 17, 2025


Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: remi <remi@mistral.ai>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Rémi Delacourt <54138269+Flechman@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>

3408e471

[V1][Frontend] Improve Shutdown And Logs (#11737) · 2b05b8ce

Robert Shaw authored Apr 16, 2025


Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

2b05b8ce

16 Apr, 2025 1 commit
- [Feature] add model aware kv ops helper (#16020) · 3ac98edc
  billishyahao authored Apr 16, 2025
```
Signed-off-by: billishyahao <bill.he@amd.com>
```
  3ac98edc
14 Apr, 2025 1 commit
- [Misc] Update usage with mooncake lib for kv transfer (#16523) · 1dd23386
  shangmingc authored Apr 14, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  1dd23386
12 Apr, 2025 1 commit
- [MISC] Make GroupCoordinator compatible with out-of-tree devices (#16464) · 68bb122e
  Huazhong Ji authored Apr 12, 2025
```
Signed-off-by: hzji210@gmail.com <hzji210@gmail.com>
```
  68bb122e
08 Apr, 2025 1 commit
- [Misc] Merge the logs of pp layers partitions (#16225) · b4ac449a
  Kebe authored Apr 08, 2025
```
Signed-off-by: Kebe <mail@kebe7jun.com>
```
  b4ac449a
03 Apr, 2025 1 commit
- [TPU] optimize the all-reduce performance (#15903) · 01b61136
  Chengji Yao authored Apr 02, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  01b61136
02 Apr, 2025 1 commit
- [CPU][Bugfix] Using custom allreduce for CPU backend (#15934) · 550b2801
  Li, Jiang authored Apr 02, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  550b2801
01 Apr, 2025 1 commit
- [Distributed] Add custom allreduce support for ROCM (#14125) · b7b7676d
  Ilya Markov authored Apr 01, 2025
```
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
```
  b7b7676d
29 Mar, 2025 2 commits
- [Feature][Disaggregated] Support XpYd disaggregated prefill with MooncakeStore (#12957) · 6fa7cd3d
  shangmingc authored Mar 29, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  6fa7cd3d
- [Misc][V1] Misc code streamlining (#15723) · 6d531ad7
  Nick Hill authored Mar 28, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  6d531ad7
28 Mar, 2025 2 commits

[Bugfix] Fix failure to launch in Tensor Parallel TP mode on macOS. (#14948) · 4e0f6076

Kebe authored Mar 28, 2025


Signed-off-by: Kebe <mail@kebe7jun.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>

4e0f6076

[TPU] Lazy Import (#15656) · bd45912b
Robert Shaw authored Mar 27, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
bd45912b

27 Mar, 2025 1 commit
- [V1] AsyncLLM data parallel (#13923) · 15dac210
  Nick Hill authored Mar 27, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  15dac210
24 Mar, 2025 3 commits
- [MISC] Refine no available block debug msg (#15076) · 9cc64514
  Yi Liu authored Mar 25, 2025
```
Signed-off-by: Yi Liu <yiliu4@habana.ai>
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Co-authored-by: Yi Liu <yiliu4@habana.ai>
```
  9cc64514
- [distributed] fix dp group (#15355) · 9606d572
  youkaichao authored Mar 24, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  9606d572
- Fix zmq IPv6 URL format error (#15341) · 038de04d
  Russell Bryant authored Mar 24, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  038de04d
20 Mar, 2025 2 commits
- [Frontend][Bugfix] support prefill decode disaggregation on deepseek (#14824) · 742369d3
  billishyahao authored Mar 20, 2025
```
Signed-off-by: billishyahao <bill.he@amd.com>
Co-authored-by: Zhai Feiyue <80079571+ZhaiFeiyue@users.noreply.github.com>
```
  742369d3
- [V1] TPU - Tensor parallel MP support (#15059) · cfbca8a2
  Alexander Matveev authored Mar 19, 2025
  
  cfbca8a2
18 Mar, 2025 1 commit

[Bugfix] torchrun compatibility (#14899) · 41491913

hoshi-hiyouga authored Mar 18, 2025


Signed-off-by: hiyouga <hiyouga@buaa.edu.cn>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>

41491913

16 Mar, 2025 1 commit
- [BugFix] Fix torch distributed stateless PG backend init (#14870) · b82662d9
  Nick Hill authored Mar 15, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  b82662d9
14 Mar, 2025 1 commit
- [Doc] Fix typo in documentation (#14783) · 3fb17d26
  yasu52 authored Mar 13, 2025
```
Signed-off-by: yasu52 <tsuguro4649@gmail.com>
```
  3fb17d26
13 Mar, 2025 1 commit

[Config][Disaggregated] Add timeout configuration for the torch.store and add... · 1bd32bc8

Mathis Felardos authored Mar 13, 2025


[Config][Disaggregated] Add timeout configuration for the torch.store and add KVTransferConfig.kv_connector_extra_config (#14367)
Signed-off-by: Mathis Felardos <mathis@mistral.ai>

1bd32bc8

11 Mar, 2025 1 commit
- [Neuron] Add Neuron device communicator for vLLM v1 (#14085) · d6123170
  gnovack authored Mar 10, 2025
  
  d6123170
09 Mar, 2025 1 commit
- [Feat] Support chunked prefill for LMCache connector (#14505) · 6d7f0377
  Jiayi Yao authored Mar 08, 2025
```
Signed-off-by: YaoJiayi <120040070@link.cuhk.edu.cn>
```
  6d7f0377
08 Mar, 2025 1 commit

[Bugfix][Disaggregated] Add a check in send_kv_caches_and_hidden_states and... · 980385f8

Mathis Felardos authored Mar 08, 2025


[Bugfix][Disaggregated] Add a check in send_kv_caches_and_hidden_states and fix the reshape of the KVCache (#14369)
Signed-off-by: Mathis Felardos <mathis@mistral.ai>

980385f8

04 Mar, 2025 1 commit
- [Security] Serialize using safetensors instead of pickle in Mooncake Pipe (#14228) · 288ca110
  Kuntai Du authored Mar 04, 2025
```
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
```
  288ca110
28 Feb, 2025 1 commit

[Bugfix][Disaggregated] patch the inflight batching on the decode node in... · b9e41734

Mathis Felardos authored Feb 28, 2025


[Bugfix][Disaggregated] patch the inflight batching on the decode node in SimpleConnector to avoid hangs in SimpleBuffer (nccl based) (#13987)
Signed-off-by: Mathis Felardos <mathis@mistral.ai>

b9e41734