Commits · 8bddb735123204872788a8ffe117321de7550e6c · OpenDAS / vllm_cscc · GitLab

12 Jan, 2025 1 commit

[Hardware][CPU] Multi-LoRA implementation for the CPU backend (#11100) · 8bddb735

Akshat Tripathi authored Jan 12, 2025


Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Oleg Mosalov <oleg@krai.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Oleg Mosalov <oleg@krai.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

8bddb735

10 Jan, 2025 1 commit
- [torch.compile] Hide KV cache behind torch.compile boundary (#11677) · cf5f000d
  Chen Zhang authored Jan 10, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  cf5f000d
09 Jan, 2025 1 commit
- [perf]fix current stream (#11870) · 310aca88
  youkaichao authored Jan 09, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  310aca88
07 Jan, 2025 1 commit
- [XPU] Make pp group initilized for pipeline-parallelism (#11648) · d93d2d74
  YiSheng5 authored Jan 07, 2025
```
Signed-off-by: yisheng <yi.sheng@intel.com>
```
  d93d2d74
06 Jan, 2025 1 commit
- [Doc][2/N] Reorganize Models and Usage sections (#11755) · ee77fdb5
  Cyrus Leung authored Jan 06, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ee77fdb5
04 Jan, 2025 1 commit
- [Core][Bugfix] Use correct device to initialize GPU data during CUDA-graph-capture (#11233) · 300acb83
  Yan Burman authored Jan 04, 2025
```
Signed-off-by: Yan Burman <yanburman@users.noreply.github.com>
Signed-off-by: Ido Asraff <idoa@atero.ai>
```
  300acb83
03 Jan, 2025 2 commits
- Update requirements-tpu.txt to support python 3.9 and 3.11 (#11695) · bf0d97d7
  Michael Goin authored Jan 03, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  bf0d97d7
- [Model] Whisper model implementation (#11280) · e1a5c2f0
  Aurick Qiao authored Jan 03, 2025
```
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
```
  e1a5c2f0
02 Jan, 2025 1 commit
- [Bugfix] Change kv scaling factor by param json on nvidia gpu (#11688) · 187e3299
  bjmsong authored Jan 03, 2025
```
Signed-off-by: bjmsong <bjmsong@126.com>
Co-authored-by: bjmsong <bjmsong@126.com>
```
  187e3299
30 Dec, 2024 1 commit
- [platforms] enable platform plugins (#11602) · b12e87f9
  youkaichao authored Dec 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  b12e87f9
24 Dec, 2024 1 commit
- [Bugfix][Hardware][CPU] Fix CPU `input_positions` creation for text-only inputs with mrope (#11434) · 7a5286cc
  Isotr0py authored Dec 24, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  7a5286cc
23 Dec, 2024 2 commits
- [Docs] Convert rST to MyST (Markdown) (#11145) · 32aa2059
  Rafael Vasquez authored Dec 23, 2024
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  32aa2059
- mypy type checking for vllm/worker (#11418) · e51719ae
  Lucas Tucker authored Dec 23, 2024
```
Signed-off-by: lucast2021 <lucast2021@headroyce.org>
Co-authored-by: lucast2021 <lucast2021@headroyce.org>
```
  e51719ae
20 Dec, 2024 1 commit
- [Misc] Add tqdm progress bar during graph capture (#11349) · b880ffb8
  Michael Goin authored Dec 19, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  b880ffb8
19 Dec, 2024 1 commit

[Model] Add JambaForSequenceClassification model (#10860) · 6c7f8815

Yehoshua Cohen authored Dec 19, 2024


Signed-off-by: Yehoshua Cohen <yehoshuaco@ai21.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Yehoshua Cohen <yehoshuaco@ai21.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

6c7f8815

16 Dec, 2024 1 commit
- [core] overhaul memory profiling and fix backward compatibility (#10511) · 551603fe
  youkaichao authored Dec 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  551603fe
13 Dec, 2024 1 commit
- [core] clean up cudagraph batchsize padding logic (#10996) · be39e3cd
  youkaichao authored Dec 12, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  be39e3cd
12 Dec, 2024 1 commit
- [Hardware][Intel-Gaudi] Enable LoRA support for Intel Gaudi (HPU) (#10565) · 81958242
  Sanju C Sudhakaran authored Dec 12, 2024
```
Signed-off-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
```
  81958242
11 Dec, 2024 6 commits
- [torch.compile] use depyf to dump torch.compile internals (#10972) · 91642db9
  youkaichao authored Dec 11, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  91642db9
- [Doc] Update docs to refer to pooling models (#11093) · cad5c0a6
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cad5c0a6
- [Misc] Split up pooling tasks (#10820) · 8f10d5e3
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f10d5e3
- [Model] PP support for Mamba-like models (#10992) · ffa48c91
  Mor Zusman authored Dec 11, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  ffa48c91
- [Misc] LoRA + Chunked Prefill (#9057) · d5c5154f
  Aurick Qiao authored Dec 10, 2024
  
  d5c5154f
- [Bugfix] Fix Mamba multistep (#11071) · 9a939737
  Tyler Michael Smith authored Dec 10, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  9a939737
07 Dec, 2024 1 commit
- [core][executor] simplify instance id (#10976) · 1b62745b
  youkaichao authored Dec 07, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  1b62745b
06 Dec, 2024 1 commit
- [ci] fix broken tests (#10956) · dcdc3faf
  youkaichao authored Dec 06, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  dcdc3faf
05 Dec, 2024 1 commit
- [Doc] Create a new "Usage" section (#10827) · aa39a8e1
  Cyrus Leung authored Dec 05, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  aa39a8e1
03 Dec, 2024 1 commit
- [torch.compile] remove compilation_context and simplify code (#10838) · dc5ce861
  youkaichao authored Dec 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  dc5ce861
02 Dec, 2024 2 commits

Fix openvino on GPU (#10793) · 9b14d978
Jani Monoses authored Dec 02, 2024

9b14d978

[Core] Implement disagg prefill by StatelessProcessGroup (#10502) · 0590ec3f

Kuntai Du authored Dec 01, 2024



This PR provides initial support for single-node disaggregated prefill in 1P1D scenario.
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: YaoJiayi <120040070@link.cuhk.edu.cn>

0590ec3f

01 Dec, 2024 1 commit
- [Misc] Rename embedding classes to pooling (#10801) · d2f058e7
  Cyrus Leung authored Dec 01, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d2f058e7
27 Nov, 2024 1 commit
- Remove hard-dependencies of Speculative decode to CUDA workers (#10587) · 0a71900b
  Chendi.Xue authored Nov 26, 2024
```
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
```
  0a71900b
25 Nov, 2024 1 commit

Support Cross encoder models (#10400) · 214efc2c

Maximilien de Bayser authored Nov 24, 2024


Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>

214efc2c

23 Nov, 2024 2 commits
- [Bugfix] `multi_modal_kwargs` broadcast for CPU tensor parallel (#10541) · 4cfe5d2b
  Isotr0py authored Nov 23, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  4cfe5d2b
- [bugfix] fix cpu tests (#10585) · d559979c
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  d559979c
22 Nov, 2024 2 commits
- [torch.compile] support all attention backends (#10558) · eebad39f
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eebad39f
- [platforms] absorb worker cls difference into platforms folder (#10555) · a111d015
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  a111d015
20 Nov, 2024 3 commits
- [TPU] Implement prefix caching for TPUs (#10307) · 2f77b6cf
  Woosuk Kwon authored Nov 20, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2f77b6cf
- [7/N] torch.compile, reduce compilation time (#10460) · 0cd3d971
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  0cd3d971
- [Hardware][CPU] Support chunked-prefill and prefix-caching on CPU (#10355) · 63f1fde2
  Li, Jiang authored Nov 20, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  63f1fde2