Commits · a28b94e6ef60b7f5aa1b97bc8d966a8d12cbc1da · OpenDAS / vllm_cscc

24 Jan, 2026 1 commit

[Performance] Split FlashAttn attention and cache update (#25954) · a28b94e6

ElizaWszola authored Jan 24, 2026


Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Luka Govedič <luka.govedic@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <luka.govedic@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>

a28b94e6

23 Jan, 2026 9 commits
- fix: Add glm4_moe_lite to MLA detection (#32614) · 586a57ad
  Markus / Mark authored Jan 23, 2026
```
Signed-off-by: marksverdhei <marksverdhei@hotmail.com>
Signed-off-by: Markus / Mark <46672778+marksverdhei@users.noreply.github.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  586a57ad
- [cudagraphs] Refactor cudagraph capture loop (#32946) · 3a414595
  Lucas Wilkinson authored Jan 23, 2026
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  3a414595
- [Model Runner V2] Add KV Connector support (#32742) · 8518b304
  Nick Hill authored Jan 23, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  8518b304
- [Bugfix][CI] Fix pre-commit (#32956) · 2d6b5371
  Matthew Bonanni authored Jan 23, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  2d6b5371
- [V1][Hybrid] Mamba Prefix Caching with align mode (#30877) · 5206e5e2
  Harry Huang authored Jan 24, 2026
```
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
```
  5206e5e2
- [Feature]: Remove DtoH Copy for lfm2_vl On Default Stream (#32815) · 13d8746c
  tianshu-Michael-yu authored Jan 23, 2026
```
Signed-off-by: Tianshu Yu <tianshuyu.formal@gmail.com>
```
  13d8746c
- [Misc] Add `get_name` to missing AttentionBackends (#32698) · 160c6fa3
  Nicolò Lucchesi authored Jan 23, 2026
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  160c6fa3
- [CI][Models] Add VLM Support for Sequence Classification Conversion (#32885) · a8eb1182
  Andreas Karatzas authored Jan 23, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
```
  a8eb1182
- [CI] Fix mypy for `vllm/v1/structured_output` (#32722) · 7ef58737
  Wentao Ye authored Jan 22, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  7ef58737
22 Jan, 2026 12 commits
- [Refactor] Remove unused tpu files (#32610) · f7448101
  Wentao Ye authored Jan 22, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  f7448101
- Add llmcompressor fp8 kv-cache quant (per-tensor and per-attn_head) (#30141) · 44f08af3
  Eldar Kurtić authored Jan 22, 2026
```
Signed-off-by: Eldar Kurtic <8884008+eldarkurtic@users.noreply.github.com>
Signed-off-by: eldarkurtic <8884008+eldarkurtic@users.noreply.github.com>
```
  44f08af3
- [Bugfix][Attention] Explicitly report support for kv_cache_dtype bfloat16 (#32795) · 955b43a5
  Matthew Bonanni authored Jan 22, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  955b43a5
- [CPU Backend] [Perf] Accelerate tensor-parallel/data-parallel inference across... · 744ef304
  Fadi Arafeh authored Jan 22, 2026
```
[CPU Backend] [Perf] Accelerate tensor-parallel/data-parallel inference across NUMA domains on Arm (#32792)
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
```
  744ef304
- [Cleanup] Move scheduler `get_routed_experts` logic to separate method (#32706) · bc14663e
  Nick Hill authored Jan 22, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  bc14663e
- [Frontend] Introduce Renderer for processing chat messages (using `ModelConfig`) (#30200) · d117a4d1
  Cyrus Leung authored Jan 22, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d117a4d1
- OffloadingConnector: Support kernel_block_size != block_size (#30692) · 421012b6
  Or Ozeri authored Jan 22, 2026
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  421012b6
- [Bugfix] Fix Whisper/encoder-decoder GPU memory leak (#32789) · ea6102b8
  Nicolò Lucchesi authored Jan 22, 2026
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  ea6102b8
- [FlashMLA] Update FlashMLA to expose new arguments (#32810) · 889722f3
  Lucas Wilkinson authored Jan 21, 2026
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  889722f3
- [EC Connector] Optimize remote cache check in scheduler (#32585) · 378385b9
  knlnguyen1802 authored Jan 22, 2026
```
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
```
  378385b9
- [Bugfix] Fix potential EAGLE spec decode segfault during graph capture (#32818) · c5487e2b
  Matt authored Jan 21, 2026
```
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
```
  c5487e2b
- [Model Runner V2] Do not error on attention backends (#32820) · 5e00b561
  Woosuk Kwon authored Jan 21, 2026
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  5e00b561
21 Jan, 2026 10 commits
- [Model Runner V2] Refactor Prompt Logprobs (#32811) · 408195ec
  Woosuk Kwon authored Jan 21, 2026
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  408195ec
- [ModelRunner V2] Don't pin reused flashinfer tensors (#32799) · 24dc30f7
  Nick Hill authored Jan 21, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  24dc30f7
- [Model Runner V2] Minor refactor for `compute_slot_mappings` (#32794) · e1da249c
  Woosuk Kwon authored Jan 21, 2026
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e1da249c
- Bump Flashinfer to v0.6.1 (#30993) · 808d6fd7
  elvischenv authored Jan 22, 2026
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
```
  808d6fd7
- [ROCm][Deepseekv3.2] Refactor Sparse Indexer as CustomOp (#29287) · 6c20e89c
  Pleaplusone authored Jan 21, 2026
```
Signed-off-by: ganyi <ygan@amd.com>
```
  6c20e89c
- [MoE Refactor] Oracle Select FP8+NVFP4 Kernels In Priority (#32414) · 42135d68
  Robert Shaw authored Jan 21, 2026
  
  42135d68
- Update FlashMLA (#32491) · b4f64e5b
  Lucas Wilkinson authored Jan 20, 2026
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  b4f64e5b
- Added qwen3 vision language moe support for speculative decoding (#32048) · 7ab80a8e
  shanjiaz authored Jan 20, 2026
```
Signed-off-by: shanjiaz <zsjwpianpian@gmail.com>
Signed-off-by: shanjiaz <43143795+shanjiaz@users.noreply.github.com>
```
  7ab80a8e
- [Cleanup] Remove unused `KVConnectorModelRunnerMixin` methods (#32077) · 6f067b1f
  Nick Hill authored Jan 20, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  6f067b1f
- OffloadingConnector: Prevent redundant loads (#29087) · 7013e9ac
  Or Ozeri authored Jan 21, 2026
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  7013e9ac
20 Jan, 2026 8 commits
- [Misc] Remove pad_for_cudagraphs from config (#30143) · 22613408
  Lucas Wilkinson authored Jan 20, 2026
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
```
  22613408
- [Bugfix] Fix byte fallback handling when using outlines (#31391) · 86c69dc5
  Shinichi Hemmi authored Jan 21, 2026
```
Signed-off-by: Shinichi Hemmi <50256998+Alnusjaponica@users.noreply.github.com>
Co-authored-by: Kenichi Maehashi <maehashi@preferred.jp>
```
  86c69dc5
- [Model Runner V2] Support FLASHINFER_MLA backend (#32709) · 9ab4388c
  Woosuk Kwon authored Jan 20, 2026
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  9ab4388c
- [Metrics] Complete removal of deprecated vllm:time_per_output_token_seconds metric (#32661) · bb917203
  杨朱 · Kiki authored Jan 20, 2026
```
This PR completes the removal of the deprecated vllm:time_per_output_token_seconds
metric that was deprecated in v0.11, hidden in v0.12, scheduled for removal in v0.13,
but delayed until v0.15.
Signed-off-by: carlory <baofa.fan@daocloud.io>
Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
```
  bb917203
- [Core] Cleanup shm based object store on engine shutdown (#32429) · 8be263c3
  Walter Beller-Morales authored Jan 20, 2026
```
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
```
  8be263c3
- [Model Runner V2] Skip kernel launch for penalties & logit_bias (#32634) · e9c83cdc
  Woosuk Kwon authored Jan 19, 2026
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e9c83cdc
- [Model] Use context managers for encoder- and LM-only mode (#32605) · 4753f3bf
  Cyrus Leung authored Jan 20, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  4753f3bf
- [Model Runner V2] Decouple temperature from penalties (#32629) · 6c01ffb8
  Woosuk Kwon authored Jan 19, 2026
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6c01ffb8