Commits · 01c22335baa03dbbc3ae662c76a089ff1c5b5742 · OpenDAS / vllm_cscc · GitLab

15 May, 2025 18 commits
- [Kernel] [V1] Fix performance regression for triton unified attention (#18161) · 01c22335
  Thomas Parnell authored May 15, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  01c22335
- add tools into TokenizeChatRequest (#18187) · 451da4bc
  hustxiayang authored May 15, 2025
```
Signed-off-by: yangxia <yangxiast@gmail.com>
```
  451da4bc
- Update deprecated type hinting in `model_loader` (#18130) · 07ad2712
  Harry Mellor authored May 15, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  07ad2712
- fix: typos (#18151) · a9944aab
  omahs authored May 15, 2025
```
Signed-off-by: omahs <73983677+omahs@users.noreply.github.com>
```
  a9944aab
- [V1] Update zmq socket creation in nixl connector (#18148) · a8f5aec2
  Russell Bryant authored May 15, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  a8f5aec2
- [CI] don't skip fixed `test_kv_cache_events()` (#18183) · de71fec8
  David Xia authored May 15, 2025
```
Signed-off-by: David Xia <david@davidxia.com>
```
  de71fec8
- [Bugfix] Fix FusedMoEPrepareAndFinalize for cuda-disalike backends (#18178) · 70f8b967
  Mengqing Cao authored May 15, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  70f8b967
- [Model] Allow the use of sliding window in Qwen2 (#17772) · dd2a9459
  inkcherry authored May 15, 2025
```
Signed-off-by: inkcherry <mingzhi.liu@intel.com>
```
  dd2a9459
- [UT] Add ut for none hash (#17892) · 420caf75
  Ning Xie authored May 15, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  420caf75
- Support custom implementations of VideoLoader backends. (#18091) · 4f07a640
  Chenheli Hua authored May 14, 2025
  
  4f07a640
- [Bugfix] Fix fp8 tests for triton_unified_attention for Triton 3.3 (#18013) · e6b8e65d
  Thomas Parnell authored May 15, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  e6b8e65d
- Update deprecated type hinting in `models` (#18132) · 26d04193
  Harry Mellor authored May 15, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  26d04193
- [Fix][ROCm] Enforce eager for all encoder-decoder models on ROCm (#18154) · 83f74c69
  Luka Govedič authored May 15, 2025
```
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
```
  83f74c69
- [Misc] add lobe-chat support (#18177) · 2dff0935
  Reid authored May 15, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  2dff0935
- [Chore] astral's ty (#18116) · afe3236e
  Aaron Pham authored May 15, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  afe3236e
- [V1][Metrics] Remove unused code (#18158) · 65334ef3
  Mark McLoughlin authored May 15, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  65334ef3
- [v1] Support multiple KV cache groups in GPU model runner (#17945) · e60f550b
  Chen Zhang authored May 15, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  e60f550b
- [Bugfix]: make most of `test_openai_schema.py` pass (#17664) · f25e0d11
  David Xia authored May 14, 2025
  
  f25e0d11
14 May, 2025 22 commits
- Upload vllm index for the rc builds (#18173) · 09f106a9
  Andrey Talman authored May 14, 2025
  
  09f106a9
- [V1] Support multiple kv connectors (#17564) · 2142035b
  Michael Goin authored May 14, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  2142035b
- [CI] Fix race condition in test_kv_cache_events test (#18169) · 78aa341d
  Russell Bryant authored May 14, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  78aa341d
- Add support for loading torchao models with `AOPerModuleConfig` (#17826) · 79747367
  Jerry Zhang authored May 14, 2025
```
Signed-off-by: Jerry Zhang <jerryzh168@gmail.com>
```
  79747367
- [V1] Structured Outputs + Thinking compatibility (#16577) · 2fc9075b
  Aaron Pham authored May 14, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
```
  2fc9075b
- [Kernel] Have rotary embeddings support tensors (#18046) · d93c976a
  Lucas Wilkinson authored May 14, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  d93c976a
- [Frontend] decrease import time of vllm.multimodal (#18031) · 749f7925
  David Xia authored May 14, 2025
```
Co-authored-by: Aaron Pham <Aaronpham0103@gmail.com>
```
  749f7925
- [CI] Disable Failing Tests (#18165) · 85686500
  Robert Shaw authored May 14, 2025
  
  85686500
- Modularize fused experts and integrate PPLX kernels (#15956) · f9c069c8
  bnellnm authored May 14, 2025
  
  f9c069c8
- [V1][Spec Decode] Share input embedding of target model with EAGLE draft model... · 418d2f8b
  Ekagra Ranjan authored May 14, 2025
```
[V1][Spec Decode] Share input embedding of target model with EAGLE draft model to free ~1GB for llama 3 model (#17326)
Co-authored-by: root <root@ekagra-8xh100.us-east5-a.c.serving-efficiency-poc.internal>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  418d2f8b
- [Doc] Update prefix cache metrics to counting tokens (#18138) · 964472b9
  Chen Zhang authored May 14, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  964472b9
- [KVConnector] Keep KVTransferParams as a dict (#18033) · 59dd311c
  Nick Hill authored May 14, 2025
  
  59dd311c
- [Bugfix] Fix chat utils tests (#18139) · d066e520
  Cyrus Leung authored May 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d066e520
- Update deprecated type hinting in `platform`, `plugins`, `triton_utils`, `vllm_flash_attn` (#18129) · c8ea982d
  Harry Mellor authored May 14, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  c8ea982d
- Update deprecated type hinting in `vllm/device_allocator` and `vllm/distributed` (#18126) · dc372b9c
  Harry Mellor authored May 14, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  dc372b9c
- Update deprecated type hinting in `vllm/lora` (#18128) · 9b5b39b6
  Harry Mellor authored May 14, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  9b5b39b6
- [doc] add missing import (#18133) · 9ccc6ded
  Reid authored May 14, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  9ccc6ded
- [Model] GritLM supports other attention backends (#18109) · d62a076e
  Cyrus Leung authored May 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d62a076e
- [Bugfix] Fix LoRA test (#18123) · 259127f8
  Jee Jee Li authored May 14, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  259127f8
- [FEAT] [ROCm]: Add AITER CK 2 Stages MoE support (#17110) · 612c2edb
  TJian authored May 14, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  612c2edb
- [Bugfix] Fix QKVCrossParallelLinear::sync_weight_attrs for PyTorch compile (#17844) · 38fe728d
  Andrzej Kotłowski authored May 14, 2025
```
Signed-off-by: Andrzej Kotłowski <akotlowski@habana.ai>
```
  38fe728d
- [Misc] replace does not exist model (#18119) · 82e7f9bb
  rongfu.leng authored May 14, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  82e7f9bb