Commits · 2142035b51795d69d7434ce74e7aad746a2bab7a · OpenDAS / vllm_cscc · GitLab

14 May, 2025 40 commits
- [V1] Support multiple kv connectors (#17564) · 2142035b
  Michael Goin authored May 14, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  2142035b
- [CI] Fix race condition in test_kv_cache_events test (#18169) · 78aa341d
  Russell Bryant authored May 14, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  78aa341d
- Add support for loading torchao models with `AOPerModuleConfig` (#17826) · 79747367
  Jerry Zhang authored May 14, 2025
```
Signed-off-by: Jerry Zhang <jerryzh168@gmail.com>
```
  79747367
- [V1] Structured Outputs + Thinking compatibility (#16577) · 2fc9075b
  Aaron Pham authored May 14, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
```
  2fc9075b
- [Kernel] Have rotary embeddings support tensors (#18046) · d93c976a
  Lucas Wilkinson authored May 14, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  d93c976a
- [Frontend] decrease import time of vllm.multimodal (#18031) · 749f7925
  David Xia authored May 14, 2025
```
Co-authored-by: Aaron Pham <Aaronpham0103@gmail.com>
```
  749f7925
- [CI] Disable Failing Tests (#18165) · 85686500
  Robert Shaw authored May 14, 2025
  
  85686500
- Modularize fused experts and integrate PPLX kernels (#15956) · f9c069c8
  bnellnm authored May 14, 2025
  
  f9c069c8
- [V1][Spec Decode] Share input embedding of target model with EAGLE draft model... · 418d2f8b
  Ekagra Ranjan authored May 14, 2025
```
[V1][Spec Decode] Share input embedding of target model with EAGLE draft model to free ~1GB for llama 3 model (#17326)
Co-authored-by: root <root@ekagra-8xh100.us-east5-a.c.serving-efficiency-poc.internal>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  418d2f8b
- [Doc] Update prefix cache metrics to counting tokens (#18138) · 964472b9
  Chen Zhang authored May 14, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  964472b9
- [KVConnector] Keep KVTransferParams as a dict (#18033) · 59dd311c
  Nick Hill authored May 14, 2025
  
  59dd311c
- [Bugfix] Fix chat utils tests (#18139) · d066e520
  Cyrus Leung authored May 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d066e520
- Update deprecated type hinting in `platform`, `plugins`, `triton_utils`, `vllm_flash_attn` (#18129) · c8ea982d
  Harry Mellor authored May 14, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  c8ea982d
- Update deprecated type hinting in `vllm/device_allocator` and `vllm/distributed` (#18126) · dc372b9c
  Harry Mellor authored May 14, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  dc372b9c
- Update deprecated type hinting in `vllm/lora` (#18128) · 9b5b39b6
  Harry Mellor authored May 14, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  9b5b39b6
- [doc] add missing import (#18133) · 9ccc6ded
  Reid authored May 14, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  9ccc6ded
- [Model] GritLM supports other attention backends (#18109) · d62a076e
  Cyrus Leung authored May 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d62a076e
- [Bugfix] Fix LoRA test (#18123) · 259127f8
  Jee Jee Li authored May 14, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  259127f8
- [FEAT] [ROCm]: Add AITER CK 2 Stages MoE support (#17110) · 612c2edb
  TJian authored May 14, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  612c2edb
- [Bugfix] Fix QKVCrossParallelLinear::sync_weight_attrs for PyTorch compile (#17844) · 38fe728d
  Andrzej Kotłowski authored May 14, 2025
```
Signed-off-by: Andrzej Kotłowski <akotlowski@habana.ai>
```
  38fe728d
- [Misc] replace does not exist model (#18119) · 82e7f9bb
  rongfu.leng authored May 14, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  82e7f9bb
- [Model] Add packed_modules_mapping for Qwen3-MOE (#18118) · 63dc3426
  Jee Jee Li authored May 14, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  63dc3426
- [Bugfix] Fix entrypoints audio test failure (#18111) · 8f5dc414
  Cyrus Leung authored May 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f5dc414
- [New Model]: support GTE NewModel (#17986) · 63ad6222
  wang.yuqi authored May 14, 2025
  
  63ad6222
- [Bugfix][Example] make lmcache v0 work. (#18051) · e7ef61c1
  majianpeng authored May 14, 2025
```
Signed-off-by: Ma, Jianpeng <jianpeng.ma@intel.com>
```
  e7ef61c1
- [Bugfix] fix moe marlin `topk_weight` loading (#18080) · d4154c35
  Jinzhen Lin authored May 14, 2025
```
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  d4154c35
- [Fix] Move "model_config" as keyword args in chat_utils.py (#18098) · 6685890d
  lkchen authored May 13, 2025
```
Signed-off-by: Linkun <github@lkchen.net>
```
  6685890d
- Fix broken example: examples/offline_inference/profiling at scheduler_config (#18117) · 33011318
  Ecthlion_zyy authored May 14, 2025
  
  33011318
- [BugFix][AMD] Compatible patch for AITER lib after 04/20 (#17912) · 4f8b3732
  qli88 authored May 14, 2025
```
Signed-off-by: Qiang Li <qiang.li2@amd.com>
```
  4f8b3732
- [AMD][torch.compile] Enable silu+fp8_quant fusion for rocm (#18082) · 7b2f28de
  Charlie Fu authored May 14, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
```
  7b2f28de
- [FEAT] [ROCm] [V1]: Add AITER biased group topk for DeepSeekV3 (#17955) · 2d912fb6
  vllmellm authored May 14, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  2d912fb6
- [Bugfix][V1] Fix FlashInfer V1 backend using the wrong VllmConfig (#18086) · 12e6c0b4
  Michael Goin authored May 13, 2025
  
  12e6c0b4
- [Bugfix] Fix FP8 Marlin MoE and enable for compressed-tensors models (#18026) · 9a2a6357
  Michael Goin authored May 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  9a2a6357
- [core][distributed] add ep group and all2all interface (#18077) · 6266c57b
  youkaichao authored May 14, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  6266c57b
- [Bug]: Fix S3 model/tokenizer path resolution (#18083) · 754b699c
  Jon Gill authored May 13, 2025
```
Signed-off-by: Jon Gill <jon@yurts.ai>
```
  754b699c
- [Misc] Remove unused numpy tensor (#18084) · 6e27c6d8
  Roger Wang authored May 13, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.me>
```
  6e27c6d8
- [P/D] Add some more debug logs to `NixlConnector` (#18102) · d5af47a1
  Nick Hill authored May 13, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  d5af47a1
- [Hardware/NVIDIA/Modelopt] Fix modelopt forward method for v1 torch.compile (#18101) · 65f0f74b
  Pavani Majety authored May 13, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  65f0f74b
- [Fix] Support CUDAGraph capture for encoder-decoder on ROCm (#18104) · 176a95c6
  Luka Govedič authored May 13, 2025
```
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
```
  176a95c6
- [v1][KVCacheManager] pass num_new_computed_tokens to kv cache manager (#18001) · f2ae883b
  Chen Zhang authored May 14, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  f2ae883b