Commits · dcdc3fafe535178037ef0a58f53607b2fb3e4190 · OpenDAS / vllm_cscc

06 Dec, 2024 1 commit
- [ci] fix broken tests (#10956) · dcdc3faf
  youkaichao authored Dec 06, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  dcdc3faf
05 Dec, 2024 1 commit
- [Doc] Create a new "Usage" section (#10827) · aa39a8e1
  Cyrus Leung authored Dec 05, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  aa39a8e1
03 Dec, 2024 1 commit
- [torch.compile] remove compilation_context and simplify code (#10838) · dc5ce861
  youkaichao authored Dec 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  dc5ce861
02 Dec, 2024 2 commits

Fix openvino on GPU (#10793) · 9b14d978
Jani Monoses authored Dec 02, 2024

9b14d978

[Core] Implement disagg prefill by StatelessProcessGroup (#10502) · 0590ec3f

Kuntai Du authored Dec 01, 2024



This PR provides initial support for single-node disaggregated prefill in 1P1D scenario.
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: YaoJiayi <120040070@link.cuhk.edu.cn>

0590ec3f

01 Dec, 2024 1 commit
- [Misc] Rename embedding classes to pooling (#10801) · d2f058e7
  Cyrus Leung authored Dec 01, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d2f058e7
27 Nov, 2024 1 commit
- Remove hard-dependencies of Speculative decode to CUDA workers (#10587) · 0a71900b
  Chendi.Xue authored Nov 26, 2024
```
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
```
  0a71900b
25 Nov, 2024 1 commit

Support Cross encoder models (#10400) · 214efc2c

Maximilien de Bayser authored Nov 24, 2024


Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>

214efc2c

23 Nov, 2024 2 commits
- [Bugfix] `multi_modal_kwargs` broadcast for CPU tensor parallel (#10541) · 4cfe5d2b
  Isotr0py authored Nov 23, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  4cfe5d2b
- [bugfix] fix cpu tests (#10585) · d559979c
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  d559979c
22 Nov, 2024 2 commits
- [torch.compile] support all attention backends (#10558) · eebad39f
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eebad39f
- [platforms] absorb worker cls difference into platforms folder (#10555) · a111d015
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  a111d015
20 Nov, 2024 3 commits
- [TPU] Implement prefix caching for TPUs (#10307) · 2f77b6cf
  Woosuk Kwon authored Nov 20, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2f77b6cf
- [7/N] torch.compile, reduce compilation time (#10460) · 0cd3d971
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  0cd3d971
- [Hardware][CPU] Support chunked-prefill and prefix-caching on CPU (#10355) · 63f1fde2
  Li, Jiang authored Nov 20, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  63f1fde2
19 Nov, 2024 1 commit
- [Platform][Refactor] Extract func `get_default_attn_backend` to `Platform` (#10358) · 8c1fb507
  Mengqing Cao authored Nov 19, 2024
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  8c1fb507
18 Nov, 2024 2 commits
- [5/N][torch.compile] torch.jit.script --> torch.compile (#10406) · 7851b451
  youkaichao authored Nov 18, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  7851b451
- [4/N][torch.compile] clean up set_torch_compile_backend (#10401) · 51bb12d1
  youkaichao authored Nov 17, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  51bb12d1
17 Nov, 2024 3 commits
- [Hardware] [HPU]add `mark_step` for hpu (#10239) · 76aab90a
  Kunshang Ji authored Nov 17, 2024
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  76aab90a
- [Bugfix][CPU] Fix CPU embedding runner with tensor parallel (#10394) · cf349c4a
  Isotr0py authored Nov 17, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  cf349c4a
- [2/N][torch.compile] make compilation cfg part of vllm cfg (#10383) · 4fd93750
  youkaichao authored Nov 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  4fd93750
16 Nov, 2024 1 commit
- [Bugfix] Fix M-RoPE position calculation when chunked prefill is enabled (#10388) · 361c29e1
  电脑星人 authored Nov 17, 2024
```
Signed-off-by: imkero <kerorek@outlook.com>
```
  361c29e1
15 Nov, 2024 1 commit
- [Bugfix] Fix fully sharded LoRA bug (#10352) · 1d65ec7e
  Jee Jee Li authored Nov 15, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  1d65ec7e
13 Nov, 2024 1 commit
- [1/N] Initial prototype for multi-modal processor (#10044) · 0b8bb86b
  Cyrus Leung authored Nov 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0b8bb86b
12 Nov, 2024 2 commits
- [Encoder Decoder] Update Mllama to run with both FlashAttention and XFormers (#9982) · b41fb9d3
  sroy745 authored Nov 12, 2024
```
Signed-off-by: Sourashis Roy <sroy@roblox.com>
```
  b41fb9d3
- [Frontend] Add per-request number of cached token stats (#10174) · 47db6ec8
  zifeitong authored Nov 12, 2024
  
  47db6ec8
11 Nov, 2024 2 commits
- [Bugfix][Hardware][CPU] Fix broken encoder-decoder CPU runner (#10218) · 2cebda42
  Isotr0py authored Nov 11, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  2cebda42
- [Hardware][CPU] Add embedding models support for CPU backend (#10193) · 58170d65
  Isotr0py authored Nov 11, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  58170d65
09 Nov, 2024 1 commit
- [0/N] Rename `MultiModalInputs` to `MultiModalKwargs` (#10040) · e0191a95
  Cyrus Leung authored Nov 09, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e0191a95
08 Nov, 2024 1 commit
- [CI/Build] Update CPU tests to include all "standard" tests (#5481) · b489fc3c
  Cyrus Leung authored Nov 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b489fc3c
07 Nov, 2024 1 commit
- [Misc] Consolidate ModelConfig code related to HF config (#10104) · db7db4aa
  Cyrus Leung authored Nov 07, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  db7db4aa
06 Nov, 2024 3 commits

[Hotfix] Fix ruff errors (#10073) · 6a585a23
Woosuk Kwon authored Nov 06, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
6a585a23

[Hardware][Intel-Gaudi] Add Intel Gaudi (HPU) inference backend (#6143) · a02a50e6

Konrad Zawora authored Nov 06, 2024


Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Bob Zhu <bob.zhu@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
Co-authored-by: Marceli Fylcek <mfylcek@habana.ai>
Co-authored-by: Himangshu Lahkar <49579433+hlahkar@users.noreply.github.com>
Co-authored-by: Vivek Goel <vgoel@habana.ai>
Co-authored-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Dominika Olszewska <dolszewska@habana.ai>
Co-authored-by: barak goldberg <149692267+bgoldberg-habana@users.noreply.github.com>
Co-authored-by: Michal Szutenberg <37601244+szutenberg@users.noreply.github.com>
Co-authored-by: Jan Kaniecki <jkaniecki@habana.ai>
Co-authored-by: Agata Dobrzynie...

a02a50e6

[Bugfix] Fix pickle of input when async output processing is on (#9931) · 966e3169
Wallas Henrique authored Nov 05, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
966e3169

05 Nov, 2024 1 commit

[Misc] Add logging for CUDA memory (#10027) · 09d35503

Chenghao (Alan) Yang authored Nov 05, 2024


Signed-off-by: Chenghao Yang <yangalan1996@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Chenghao Yang <yangalan1996@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>

09d35503

02 Nov, 2024 3 commits
- [3/N] model runner pass the whole config to model (#9958) · cea808f3
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  cea808f3
- [2/N] executor pass the complete config to worker/modelrunner (#9938) · e8937954
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  e8937954
- [Encoder Decoder] Add flash_attn kernel support for encoder-decoder models (#9559) · a78dd330
  sroy745 authored Nov 01, 2024
  
  a78dd330
01 Nov, 2024 1 commit
- [Core][VLM] Add precise multi-modal placeholder tracking (#8346) · 6c0b7f54
  Peter Salas authored Nov 01, 2024
```
Signed-off-by: Peter Salas <peter@fixie.ai>
```
  6c0b7f54
31 Oct, 2024 1 commit
- [Misc] Remove deprecated arg for cuda graph capture (#9864) · 3ea2dc2e
  Roger Wang authored Oct 31, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  3ea2dc2e