Commits · db100c5cdebc7140b57cbb40b20b5a28d7bff386 · OpenDAS / vllm_cscc · GitLab

22 Nov, 2024 1 commit
- [platforms] absorb worker cls difference into platforms folder (#10555) · a111d015
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  a111d015
20 Nov, 2024 3 commits
- [TPU] Implement prefix caching for TPUs (#10307) · 2f77b6cf
  Woosuk Kwon authored Nov 20, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2f77b6cf
- [7/N] torch.compile, reduce compilation time (#10460) · 0cd3d971
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  0cd3d971
- [Hardware][CPU] Support chunked-prefill and prefix-caching on CPU (#10355) · 63f1fde2
  Li, Jiang authored Nov 20, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  63f1fde2
19 Nov, 2024 1 commit
- [Platform][Refactor] Extract func `get_default_attn_backend` to `Platform` (#10358) · 8c1fb507
  Mengqing Cao authored Nov 19, 2024
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  8c1fb507
18 Nov, 2024 2 commits
- [5/N][torch.compile] torch.jit.script --> torch.compile (#10406) · 7851b451
  youkaichao authored Nov 18, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  7851b451
- [4/N][torch.compile] clean up set_torch_compile_backend (#10401) · 51bb12d1
  youkaichao authored Nov 17, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  51bb12d1
17 Nov, 2024 3 commits
- [Hardware] [HPU]add `mark_step` for hpu (#10239) · 76aab90a
  Kunshang Ji authored Nov 17, 2024
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  76aab90a
- [Bugfix][CPU] Fix CPU embedding runner with tensor parallel (#10394) · cf349c4a
  Isotr0py authored Nov 17, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  cf349c4a
- [2/N][torch.compile] make compilation cfg part of vllm cfg (#10383) · 4fd93750
  youkaichao authored Nov 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  4fd93750
16 Nov, 2024 1 commit
- [Bugfix] Fix M-RoPE position calculation when chunked prefill is enabled (#10388) · 361c29e1
  电脑星人 authored Nov 17, 2024
```
Signed-off-by: imkero <kerorek@outlook.com>
```
  361c29e1
15 Nov, 2024 1 commit
- [Bugfix] Fix fully sharded LoRA bug (#10352) · 1d65ec7e
  Jee Jee Li authored Nov 15, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  1d65ec7e
13 Nov, 2024 1 commit
- [1/N] Initial prototype for multi-modal processor (#10044) · 0b8bb86b
  Cyrus Leung authored Nov 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0b8bb86b
12 Nov, 2024 2 commits
- [Encoder Decoder] Update Mllama to run with both FlashAttention and XFormers (#9982) · b41fb9d3
  sroy745 authored Nov 12, 2024
```
Signed-off-by: Sourashis Roy <sroy@roblox.com>
```
  b41fb9d3
- [Frontend] Add per-request number of cached token stats (#10174) · 47db6ec8
  zifeitong authored Nov 12, 2024
  
  47db6ec8
11 Nov, 2024 2 commits
- [Bugfix][Hardware][CPU] Fix broken encoder-decoder CPU runner (#10218) · 2cebda42
  Isotr0py authored Nov 11, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  2cebda42
- [Hardware][CPU] Add embedding models support for CPU backend (#10193) · 58170d65
  Isotr0py authored Nov 11, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  58170d65
09 Nov, 2024 1 commit
- [0/N] Rename `MultiModalInputs` to `MultiModalKwargs` (#10040) · e0191a95
  Cyrus Leung authored Nov 09, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e0191a95
08 Nov, 2024 1 commit
- [CI/Build] Update CPU tests to include all "standard" tests (#5481) · b489fc3c
  Cyrus Leung authored Nov 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b489fc3c
07 Nov, 2024 1 commit
- [Misc] Consolidate ModelConfig code related to HF config (#10104) · db7db4aa
  Cyrus Leung authored Nov 07, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  db7db4aa
06 Nov, 2024 3 commits

[Hotfix] Fix ruff errors (#10073) · 6a585a23
Woosuk Kwon authored Nov 06, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
6a585a23

[Hardware][Intel-Gaudi] Add Intel Gaudi (HPU) inference backend (#6143) · a02a50e6

Konrad Zawora authored Nov 06, 2024


Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Bob Zhu <bob.zhu@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
Co-authored-by: Marceli Fylcek <mfylcek@habana.ai>
Co-authored-by: Himangshu Lahkar <49579433+hlahkar@users.noreply.github.com>
Co-authored-by: Vivek Goel <vgoel@habana.ai>
Co-authored-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Dominika Olszewska <dolszewska@habana.ai>
Co-authored-by: barak goldberg <149692267+bgoldberg-habana@users.noreply.github.com>
Co-authored-by: Michal Szutenberg <37601244+szutenberg@users.noreply.github.com>
Co-authored-by: Jan Kaniecki <jkaniecki@habana.ai>
Co-authored-by: Agata Dobrzyniewicz <160237065+adobrzyniewicz-habana@users.noreply.github.com>
Co-authored-by: Krzysztof Wisniewski <kwisniewski@habana.ai>
Co-authored-by: Dudi Lester <160421192+dudilester@users.noreply.github.com>
Co-authored-by: Ilia Taraban <tarabanil@gmail.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Michał Kuligowski <mkuligowski@habana.ai>
Co-authored-by: Jakub Maksymczuk <jmaksymczuk@habana.ai>
Co-authored-by: Tomasz Zielinski <85164140+tzielinski-habana@users.noreply.github.com>
Co-authored-by: Sun Choi <schoi@habana.ai>
Co-authored-by: Iryna Boiko <iboiko@habana.ai>
Co-authored-by: Bob Zhu <41610754+czhu15@users.noreply.github.com>
Co-authored-by: hlin99 <73271530+hlin99@users.noreply.github.com>
Co-authored-by: Zehao Huang <zehao.huang@intel.com>
Co-authored-by: Andrzej Kotłowski <Andrzej.Kotlowski@intel.com>
Co-authored-by: Yan Tomsinsky <73292515+Yantom1@users.noreply.github.com>
Co-authored-by: Nir David <ndavid@habana.ai>
Co-authored-by: Yu-Zhou <yu.zhou@intel.com>
Co-authored-by: Ruheena Suhani Shaik <rsshaik@habana.ai>
Co-authored-by: Karol Damaszke <kdamaszke@habana.ai>
Co-authored-by: Marcin Swiniarski <mswiniarski@habana.ai>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Jacek Czaja <jacek.czaja@intel.com>
Co-authored-by: Jacek Czaja <jczaja@habana.ai>
Co-authored-by: Yuan <yuan.zhou@outlook.com>

a02a50e6

[Bugfix] Fix pickle of input when async output processing is on (#9931) · 966e3169
Wallas Henrique authored Nov 05, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
966e3169

05 Nov, 2024 1 commit

[Misc] Add logging for CUDA memory (#10027) · 09d35503

Chenghao (Alan) Yang authored Nov 05, 2024


Signed-off-by: Chenghao Yang <yangalan1996@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Chenghao Yang <yangalan1996@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>

09d35503

02 Nov, 2024 3 commits
- [3/N] model runner pass the whole config to model (#9958) · cea808f3
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  cea808f3
- [2/N] executor pass the complete config to worker/modelrunner (#9938) · e8937954
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  e8937954
- [Encoder Decoder] Add flash_attn kernel support for encoder-decoder models (#9559) · a78dd330
  sroy745 authored Nov 01, 2024
  
  a78dd330
01 Nov, 2024 1 commit
- [Core][VLM] Add precise multi-modal placeholder tracking (#8346) · 6c0b7f54
  Peter Salas authored Nov 01, 2024
```
Signed-off-by: Peter Salas <peter@fixie.ai>
```
  6c0b7f54
31 Oct, 2024 1 commit
- [Misc] Remove deprecated arg for cuda graph capture (#9864) · 3ea2dc2e
  Roger Wang authored Oct 31, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  3ea2dc2e
30 Oct, 2024 2 commits
- [TPU] Correctly profile peak memory usage & Upgrade PyTorch XLA (#9438) · 211fe91a
  Woosuk Kwon authored Oct 30, 2024
  
  211fe91a
- [Bugfix] Fix multi nodes TP+PP for XPU (#8884) · 04a3ae0a
  Yan Ma authored Oct 30, 2024
```
Signed-off-by: YiSheng5 <syhm@mail.ustc.edu.cn>
Signed-off-by: yan ma <yan.ma@intel.com>
Co-authored-by: YiSheng5 <syhm@mail.ustc.edu.cn>
```
  04a3ae0a
28 Oct, 2024 1 commit
- [Hardware][ROCM] using current_platform.is_rocm (#9642) · 4e2d95e3
  wangshuai09 authored Oct 28, 2024
```
Signed-off-by: wangshuai09 <391746016@qq.com>
```
  4e2d95e3
27 Oct, 2024 1 commit
- [core] cudagraph output with tensor weak reference (#9724) · 8549c826
  youkaichao authored Oct 27, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  8549c826
26 Oct, 2024 1 commit
- [Hardware][openvino] is_openvino --> current_platform.is_openvino (#9716) · 5cbdccd1
  Mengqing Cao authored Oct 26, 2024
  
  5cbdccd1
23 Oct, 2024 1 commit
- [Hardware][XPU] using current_platform.is_xpu (#9605) · 2394962d
  Mengqing Cao authored Oct 23, 2024
  
  2394962d
21 Oct, 2024 2 commits
- [Bugfix][Misc]: fix graph capture for decoder (#9549) · 8ca89548
  yudian0504 authored Oct 22, 2024
  
  8ca89548
- [Doc] Consistent naming of attention backends (#9498) · 496e991d
  Thomas Parnell authored Oct 21, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  496e991d
20 Oct, 2024 1 commit
- [Kernel] Support sliding window in flash attention backend (#9403) · 4fa3e333
  Chen Zhang authored Oct 20, 2024
  
  4fa3e333
19 Oct, 2024 1 commit
- 🐛 fix torch memory profiling (#9516) · 380e1863
  Joe Runde authored Oct 18, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  380e1863
18 Oct, 2024 1 commit
- [Model] Add user-configurable task for models that support both generation and embedding (#9424) · 051eaf6d
  Cyrus Leung authored Oct 19, 2024
  
  051eaf6d