Commits · 2ce3d0ce360bf1e31e7ea3b2b0971a37bf5707c6 · OpenDAS / vllm_cscc

02 Apr, 2026 1 commit

[Feature] KV cache per-token-head INT8/FP8 quantization (#38378) · 2ce3d0ce

JartX authored Apr 02, 2026


Signed-off-by: JartX <sagformas@epdcenter.es>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: yangyang4991 <yangyang4991@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

2ce3d0ce

01 Apr, 2026 4 commits

[Misc] Rename think_start_str/think_end_str to reasoning_start_str/reasoning_end_str (#38242) · cbe7d180
Chauncey authored Apr 02, 2026
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
cbe7d180
[1/N][Cleanup] Standardize on use of `is_quantized_kv_cache` (#38659) · 116f4be4
Matthew Bonanni authored Apr 01, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
116f4be4

[vLLM IR] 1/N Implement IR skeleton and rms_norm op (#33825) · 40bb1750

Luka Govedič authored Mar 31, 2026


Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
Signed-off-by: chzhang <chaojun.zhang@intel.com>
Signed-off-by: Luka Govedic <luka.govedic@gmail.com>
Co-authored-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Luka Govedič <ProExpertProg@h100-01.nemg-001.lab.rdu2.dc.redhat.com>

40bb1750

[Feat][v1] Simple yet General CPU KV Cache Offloading (#37160) · 91e4521f
Yifan Qiao authored Mar 31, 2026
```
Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
```
91e4521f

31 Mar, 2026 3 commits

replace cuda_device_count_stateless() to current_platform.device_count() (#37841) · 4dfad17e

wliao2 authored Mar 31, 2026


Signed-off-by: Liao, Wei <wei.liao@intel.com>
Signed-off-by: wliao2 <wei.liao@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>

4dfad17e

[EPLB] Add alternative communication for EPLB weight exchange (#33176) · abdbb683

Ilya Markov authored Mar 31, 2026


Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Markov Ilya <markovilya19@gmail.com>
Co-authored-by: Markov Ilya <markovilya19@gmail.com>

abdbb683

DOC: TPU mention fix (#38129) · 8278825b
Mateusz Sokół authored Mar 31, 2026
```
Signed-off-by: Mateusz Sokół <mat646@gmail.com>
```
8278825b

30 Mar, 2026 8 commits
- [EPLB] Optmize eplb mapping and record in router for prefill (#36261) · 12701e8a
  Ilya Markov authored Mar 30, 2026
```
Signed-off-by: ilmarkov <markovilya197@gmail.com>
```
  12701e8a
- [Feat][Spec Decode] DFlash (#36847) · 494636b2
  Benjamin Chislett authored Mar 30, 2026
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
  494636b2
- [Bugfix][MLA] Change default SM100 MLA prefill backend back to TRT-LLM (#38562) · 2c734ed0
  Matthew Bonanni authored Mar 30, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  2c734ed0
- [HMA]Fix corner case when hybrid page_size can not be evenly divided issue... · 3b1dbaad
  Chendi.Xue authored Mar 30, 2026
```
[HMA]Fix corner case when hybrid page_size can not be evenly divided issue (blk_size=64,tp=4) (#37467)
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
```
  3b1dbaad
- [Mamba] Add stochastic rounding support (#35753) · 8e6293e8
  roikoren755 authored Mar 30, 2026
```
Signed-off-by: Roi Koren <roik@nvidia.com>
```
  8e6293e8
- [ROCm][Bugfix] fix exception related to trust_remote_code for MiniMax-M2.1-MXFP4 (#37698) · dbdd9ae0
  Hongxia Yang authored Mar 30, 2026
```
Signed-off-by: Hongxia Yang <hongxiay.yang@amd.com>
Co-authored-by: Hongxia Yang <hongxiay.yang@amd.com>
```
  dbdd9ae0
- [Mamba][Bugfix] Raise on insufficient cache blocks instead of silently capping... · cc06b4e8
  Nicolò Lucchesi authored Mar 30, 2026
```
[Mamba][Bugfix] Raise on insufficient cache blocks instead of silently capping cudagraph sizes (#38270)
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  cc06b4e8
- [Model][Quantization] Add GGUF support for MiniMax-M2.1 (#36965) · 63babd17
  PikaPikachu authored Mar 30, 2026
```
Signed-off-by: kangletian <Letian.Kang@amd.com>
```
  63babd17
28 Mar, 2026 1 commit
- [Bugfix] Disallow renderer_num_workers > 1 with mm processor cache (#38418) · bda3eda8
  Bvicii authored Mar 28, 2026
```
Signed-off-by: Bvicii <yizhanhuang2002@gmail.com>
```
  bda3eda8
27 Mar, 2026 4 commits

{ROCm]: gpt-oss fusion/padding fixes (#38043) · 0e9358c1

Rohan Potdar authored Mar 27, 2026


Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: Rohan Potdar <66227218+Rohan138@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>

0e9358c1

Remove need for explicit `\n` in docstring lists for `--help` formatting (#38350) · 21d2b53f

Harry Mellor authored Mar 27, 2026


Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

21d2b53f

enable skipping of SW attention layers when using FP8 KV cache (#33695) · 98e7f223
Jonas M. Kübler authored Mar 27, 2026
```
Signed-off-by: Jonas Kuebler <kuebj@amazon.com>
```
98e7f223
[Bugfix] Offload blocking tokenizer ops to shared thread pool to unblock event loop (#34789) · 999dfc16
Bvicii authored Mar 26, 2026
```
Signed-off-by: Bvicii <yizhanhuang2002@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
999dfc16

26 Mar, 2026 6 commits
- [Model Runner V2] Enable forcing a specific acceptance rate during rejection sampling (#38045) · c32e9760
  Giancarlo Delfin authored Mar 26, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  c32e9760
- [Renderer] Consolidate factory methods (#38218) · 2e225f7b
  Cyrus Leung authored Mar 26, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  2e225f7b
- DOC: Documentation pages fixes (#38125) · b1cb1d3d
  Mateusz Sokół authored Mar 26, 2026
```
Signed-off-by: Mateusz Sokół <mat646@gmail.com>
```
  b1cb1d3d
- [Bugfix] Fix DeepGemm E8M0 accuracy degradation for Qwen3.5 FP8 on Blackwell (#38083) · 52069012
  Vadim Gimpelson authored Mar 26, 2026
```
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
```
  52069012
- [Model] Add torch.compile support for InternVL vision encoder (#38049) · 38de8223
  Terry Gao authored Mar 25, 2026
```
Signed-off-by: tianrengao <terrygao87@gmail.com>
```
  38de8223
- Various Transformers v5 fixes (#38127) · 3c3c0842
  Harry Mellor authored Mar 26, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  3c3c0842
25 Mar, 2026 5 commits
- [ROCm]: Update rope+kvcache fusion conditions and disable custom op by default (#36716) · a0e8c740
  Rohan Potdar authored Mar 25, 2026
```
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
```
  a0e8c740
- [Feature] Support per-draft-model MoE backend via `--speculative-config` (#37880) · cd764301
  Andrii Skliar authored Mar 25, 2026
```
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: [Andrii Skliar] <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
```
  cd764301
- [Mypy] Better fixes for the `mypy` issues in `vllm/config` (#37902) · d215d1ef
  Harry Mellor authored Mar 25, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d215d1ef
- Better weight tying check for multimodal models (#38035) · 1e673a43
  Harry Mellor authored Mar 25, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  1e673a43
- [Performance] Auto-enable prefetch on NFS with RAM guard (#37673) · a93a53f8
  Artem Perevedentsev authored Mar 25, 2026
```
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
```
  a93a53f8
24 Mar, 2026 5 commits

[Perf] Disable inductor runtime asserts by default for serving perfor… (#37485) · 82580b10
Terry Gao authored Mar 24, 2026
```
Signed-off-by: tianrengao <terrygao87@gmail.com>
Co-authored-by: Tianren Gao <tianren@fb.com>
```
82580b10
[Bugfix] Pass hf_token through config loading paths for gated model support (#37920) · 54b0578a
Javier De Jesus authored Mar 24, 2026
```
Signed-off-by: javierdejesusda <javier.dejesusj9@gmail.com>
```
54b0578a

[Core] add option to schedule requests based on full ISL (#37307) · dc78c2c9

Dan Blanaru authored Mar 24, 2026


Signed-off-by: Dan Blanaru <48605845+DanBlanaru@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>

dc78c2c9

[Feature] limit thinking tokens (hard limit) (#20859) · 47318847

Sungjae Lee authored Mar 25, 2026


Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Signed-off-by: Sungjae Lee <sung-jae.lee@navercorp.com>
Signed-off-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

47318847

[Deprecate] Deprecate pooling multi task support. (#37956) · 1b6cb920

wang.yuqi authored Mar 24, 2026


Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

1b6cb920

23 Mar, 2026 3 commits

[Bugfix] Register VLLM_BATCH_INVARIANT in envs.py to fix spurious unknown env var warning (#35007) · dc6908ac

Ranran authored Mar 23, 2026


Signed-off-by: Ranran <1012869439@qq.com>
Signed-off-by: Ranran <hzz5361@psu.edu>
Signed-off-by: ran <hzz5361@psu.edu>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

dc6908ac

[Async][Spec Decoding] Zero-bubble async scheduling + spec decoding (#32951) · fafe76b4

Matthew Bonanni authored Mar 23, 2026


Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Co-authored-by: zhrrr <43847754+izhuhaoran@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>

fafe76b4

[Mypy] Fix mypy for `vllm/config` (#37808) · 45bd5c8e
Wentao Ye authored Mar 23, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
45bd5c8e