Commits · aba8d6ee006b78149ac4514f460e4038b2d4f607 · OpenDAS / vllm_cscc · GitLab

08 Jan, 2025 1 commit
- [Doc] Move examples into categories (#11840) · aba8d6ee
  Harry Mellor authored Jan 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  aba8d6ee
07 Jan, 2025 1 commit
- fix init error for MessageQueue when n_local_reader is zero (#11768) · e512f76a
  XiaobingZhang authored Jan 07, 2025
  
  e512f76a
06 Jan, 2025 1 commit
- [distributed] remove pynccl's redundant change_state (#11749) · 9e764e7b
  cennn authored Jan 06, 2025
  
  9e764e7b
05 Jan, 2025 1 commit
- [distributed] remove pynccl's redundant stream (#11744) · 635b8972
  cennn authored Jan 05, 2025
  
  635b8972
04 Jan, 2025 1 commit
- [Core][Bugfix] Use correct device to initialize GPU data during CUDA-graph-capture (#11233) · 300acb83
  Yan Burman authored Jan 04, 2025
```
Signed-off-by: Yan Burman <yanburman@users.noreply.github.com>
Signed-off-by: Ido Asraff <idoa@atero.ai>
```
  300acb83
30 Dec, 2024 1 commit
- [platforms] enable platform plugins (#11602) · b12e87f9
  youkaichao authored Dec 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  b12e87f9
29 Dec, 2024 1 commit
- [Misc] KV cache transfer connector registry (#11481) · faef77c0
  Kuntai Du authored Dec 29, 2024
```
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
```
  faef77c0
15 Dec, 2024 1 commit
- [Core] Support disaggregated prefill with Mooncake Transfer Engine (#10884) · d263bd9d
  shangmingc authored Dec 16, 2024
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  d263bd9d
12 Dec, 2024 2 commits
- [Bugfix] Fix value unpack error of simple connector for KVCache transfer. (#11058) · db6c264a
  shangmingc authored Dec 13, 2024
```
Signed-off-by: ShangmingCai <csmthu@gmail.com>
```
  db6c264a
- [core][distributed] initialization from StatelessProcessGroup (#10986) · 62de37a3
  youkaichao authored Dec 12, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  62de37a3
10 Dec, 2024 1 commit
- [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) · 28b3a1c7
  Tyler Michael Smith authored Dec 10, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  28b3a1c7
03 Dec, 2024 1 commit
- [core][distributed] add pynccl broadcast (#10843) · 21fe7b48
  youkaichao authored Dec 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  21fe7b48
02 Dec, 2024 1 commit

[Core] Implement disagg prefill by StatelessProcessGroup (#10502) · 0590ec3f

Kuntai Du authored Dec 01, 2024



This PR provides initial support for single-node disaggregated prefill in 1P1D scenario.
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: YaoJiayi <120040070@link.cuhk.edu.cn>

0590ec3f

26 Nov, 2024 1 commit

custom allreduce + torch.compile (#10121) · 9a88f897

Sage Moore authored Nov 26, 2024


Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>

9a88f897

23 Nov, 2024 1 commit
- [Misc] Add pynccl wrappers for all_gather and reduce_scatter (#9432) · 978b3974
  Tyler Michael Smith authored Nov 22, 2024
  
  978b3974
13 Nov, 2024 1 commit
- [core][distributed] use tcp store directly (#10275) · 0d4ea3fb
  youkaichao authored Nov 12, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  0d4ea3fb
11 Nov, 2024 1 commit
- [core][distributed] add stateless process group (#10216) · e6de9784
  youkaichao authored Nov 11, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  e6de9784
08 Nov, 2024 1 commit
- [Bugfix][XPU] Fix xpu tp by introducing XpuCommunicator (#10144) · f10797c0
  Yan Ma authored Nov 08, 2024
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  f10797c0
07 Nov, 2024 2 commits
- [Core][Distributed] Refactor ipc buffer init in CustomAllreduce (#10030) · 6192e9b8
  Hanzhi Zhou authored Nov 06, 2024
```
Signed-off-by: Hanzhi Zhou <hanzhi713@gmail.com>
```
  6192e9b8
- [core][distributed] add stateless_init_process_group (#10072) · 719c1ca4
  youkaichao authored Nov 06, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  719c1ca4
06 Nov, 2024 4 commits

[CI/Build] Drop Python 3.8 support (#10038) · 098f94de

Russell Bryant authored Nov 06, 2024


Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

098f94de

[Hardware][Intel-Gaudi] Add Intel Gaudi (HPU) inference backend (#6143) · a02a50e6

Konrad Zawora authored Nov 06, 2024


Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Bob Zhu <bob.zhu@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
Co-authored-by: Marceli Fylcek <mfylcek@habana.ai>
Co-authored-by: Himangshu Lahkar <49579433+hlahkar@users.noreply.github.com>
Co-authored-by: Vivek Goel <vgoel@habana.ai>
Co-authored-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Dominika Olszewska <dolszewska@habana.ai>
Co-authored-by: barak goldberg <149692267+bgoldberg-habana@users.noreply.github.com>
Co-authored-by: Michal Szutenberg <37601244+szutenberg@users.noreply.github.com>
Co-authored-by: Jan Kaniecki <jkaniecki@habana.ai>
Co-authored-by: Agata Dobrzynie...

a02a50e6

[CI/Build] drop support for Python 3.8 EOL (#8464) · 21063c11
Aaron Pham authored Nov 06, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
21063c11
[distributed] add function to create ipc buffers directly (#10064) · 4be3a451
youkaichao authored Nov 05, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
4be3a451

05 Nov, 2024 1 commit
- [Core] Use os.sched_yield in ShmRingBuffer instead of time.sleep (#9994) · 04bbf38e
  Tyler Michael Smith authored Nov 04, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  04bbf38e
01 Nov, 2024 1 commit
- [torch.compile] directly register custom op (#9896) · 96e0c9cb
  youkaichao authored Oct 31, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  96e0c9cb
30 Oct, 2024 2 commits
- [Bugfix] Fix multi nodes TP+PP for XPU (#8884) · 04a3ae0a
  Yan Ma authored Oct 30, 2024
```
Signed-off-by: YiSheng5 <syhm@mail.ustc.edu.cn>
Signed-off-by: yan ma <yan.ma@intel.com>
Co-authored-by: YiSheng5 <syhm@mail.ustc.edu.cn>
```
  04a3ae0a
- [core][distributed] fix custom allreduce in pytorch 2.5 (#9815) · 1ab6f6b4
  youkaichao authored Oct 29, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  1ab6f6b4
24 Oct, 2024 1 commit
- [torch.compile] expanding support and fix allgather compilation (#9637) · ad6f7805
  Yongzao authored Oct 24, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  ad6f7805
22 Oct, 2024 1 commit
- [Hardware][CPU] using current_platform.is_cpu (#9536) · 3ddbe255
  wangshuai09 authored Oct 22, 2024
  
  3ddbe255
18 Oct, 2024 1 commit
- [MISC] Consolidate cleanup() and refactor offline_inference_with_prefix.py (#9510) · d11bf435
  Cody Yu authored Oct 18, 2024
  
  d11bf435
04 Oct, 2024 1 commit
- [torch.compile] improve allreduce registration (#9061) · 663874e0
  youkaichao authored Oct 04, 2024
  
  663874e0
27 Sep, 2024 1 commit
- [misc][distributed] add VLLM_SKIP_P2P_CHECK flag (#8911) · 18e60d7d
  youkaichao authored Sep 27, 2024
  
  18e60d7d
23 Sep, 2024 1 commit
- [Core] Allow IPv6 in VLLM_HOST_IP with zmq (#8575) · b05f5c92
  Russell Bryant authored Sep 23, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  b05f5c92
21 Sep, 2024 1 commit
- [MISC] add support custom_op check (#8557) · d4bf085a
  Kunshang Ji authored Sep 21, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  d4bf085a
18 Sep, 2024 2 commits
- [Core] zmq: bind only to 127.0.0.1 for local-only usage (#8543) · d65798f7
  Russell Bryant authored Sep 18, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  d65798f7
- [CI/Build] Avoid CUDA initialization (#8534) · 6ffa3f31
  Cyrus Leung authored Sep 18, 2024
  
  6ffa3f31
17 Sep, 2024 1 commit
- [torch.compile] register allreduce operations as custom ops (#8526) · 99aa4edd
  youkaichao authored Sep 16, 2024
  
  99aa4edd
30 Aug, 2024 1 commit
- [TPU] Support single and multi-host TPUs on GKE (#7613) · 2148441f
  Richard Liu authored Aug 30, 2024
  
  2148441f
26 Aug, 2024 1 commit
- [misc] fix custom allreduce p2p cache file generation (#7853) · 05826c88
  youkaichao authored Aug 26, 2024
  
  05826c88