Commits · c8c3935b701380aee7494f7f5021e8e06de2d9b7 · OpenDAS / vllm_cscc

04 Mar, 2026 24 commits

[Bugfix][Model] Fix FP8 k_scale/v_scale not loaded for Qwen3-MoE (#35656) · c8c3935b
Raghavan authored Mar 04, 2026
```
Signed-off-by: raghavan <oneraghavan@gmail.com>
```
c8c3935b

[Bugfix][CPUOffloadingManager] Prevent eviction of already-stored blocks in... · bb6888b8

Ronen Schaffer authored Mar 04, 2026


[Bugfix][CPUOffloadingManager] Prevent eviction of already-stored blocks in LRU/ARC `prepare_store()` (#35846)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>

bb6888b8

[MISC] fixed tool_parser mypy errors (#35640) · 1aaec59d

Taneem Ibrahim authored Mar 04, 2026

Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

1aaec59d

[Feature] Add basic metrics for /realtime endpoint (#35500) · 1659b2e0

pougetat authored Mar 04, 2026

Signed-off-by: Thomas Pouget-Abadie <thomaspou@microsoft.com>
Signed-off-by: pougetat <thomas.pougetabadie@gmail.com>
Co-authored-by: Thomas Pouget-Abadie <thomaspou@microsoft.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

1659b2e0

[Bugfix] Cap FULL decode cudagraph sizes for Mamba/hybrid models (#34094) (#34571) · d6e04f4c
haosdent authored Mar 04, 2026
```
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
```
d6e04f4c
[XPU] bump vllm-xpu-kernels to v0.1.3 (#35984) · a8f66cbd
Kunshang Ji authored Mar 04, 2026
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
a8f66cbd

[Hardware] Replace `torch.cuda.empty_cache` with `torch.accelerator.empty_cache` (#30681) · 16d2ad1d

Kunshang Ji authored Mar 04, 2026


Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

16d2ad1d

[ROCm][Bugfix] Fall back from CK MXFP4 MoE when GEMM dimensions are unsupported (#35893) · 5dc35387
Chuan (Richard) Li authored Mar 04, 2026
```
Signed-off-by: Li <chuali@amd.com>
```
5dc35387

[Bugfix] Add missing dynamic_arg_dims for Qwen3-ASR torch.compile (#35869) · 36bf2131

Nathan Price authored Mar 04, 2026


Signed-off-by: Nathan Price <nathan@abridge.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

36bf2131

[Core] Remove busy loop from idle buffer readers (#28053) · 6f0dd938

Joe Runde authored Mar 04, 2026


Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>

6f0dd938

Support Audio Extraction from MP4 Video for Nemotron Nano VL (#35539) · 5d199ac8

Andrii Skliar authored Mar 04, 2026


Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Andrii <askliar@nvidia.com>
Co-authored-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: Andrii Skliar <askliar@oci-nrt-cs-001-vscode-01.cm.cluster>
Co-authored-by: Andrii <askliar@nvidia.com>
Co-authored-by: root <root@pool0-03748.cm.cluster>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: root <root@pool0-02416.cm.cluster>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: root <root@pool0-04880.cm.cluster>

5d199ac8

[cohere][fix][spec-decode]: fix crash when allowed_token_ids is set without penalties (#35654) · 9e0f44be
Komal Kumar Teru authored Mar 04, 2026
```
Signed-off-by: kkt-cohere <komal@cohere.com>
```
9e0f44be
[Bugfix] Improve engine ready timeout error message (#35616) · 097eb544
lailoo authored Mar 04, 2026
```
Signed-off-by: damaozi <1811866786@qq.com>
```
097eb544
[BugFix] Support tool_choice=none in the Anthropic API (#35835) · 7cdba98e
ShiJie Zhong authored Mar 04, 2026
```
Signed-off-by: ZhongsJie <zhongsjie@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
```
7cdba98e
[Rocm][CI] Fix ROCm LM Eval Large Models (8 Card) (#35913) · 3c85cd9d
Charlie Fu authored Mar 03, 2026
```
Signed-off-by: charlifu <charlifu@amd.com>
```
3c85cd9d
[Bugfix] Guard mm_token_type_ids kwarg in get_mrope_input_positions (#35711) · edba1504
Andreas Karatzas authored Mar 03, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
```
edba1504
[Refactor] Clean up processor kwargs extraction (#35872) · e3793961
Cyrus Leung authored Mar 04, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
e3793961
[Chore] Remove debug code in model implementation (#35883) · 6e9f21e8
Isotr0py authored Mar 04, 2026
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
6e9f21e8

[model] support FireRedASR2 (#35727) · c1d96340

AllenDou authored Mar 04, 2026


Signed-off-by: zixiao <shunli.dsl@alibaba-inc.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: zixiao <shunli.dsl@alibaba-inc.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

c1d96340

[PluggableLayer][MM] Add PluggableLayer for RelPosAttention (#33753) · 77e6dcbb
Shanshan Shen authored Mar 04, 2026
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
77e6dcbb
[Bugfix] Fix EVS implementation for Qwen3 VL (#33607) · 70c73df6
William Zhang authored Mar 03, 2026
```
Signed-off-by: 2ez4bz <133824995+2ez4bz@users.noreply.github.com>
```
70c73df6

Enable bnb for multiple indices weight (#35838) · 9a9d4424

xjx authored Mar 04, 2026


Signed-off-by: xjx <493337577@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

9a9d4424

[ROCm][CI] Support async weight transfer example with platform-aware determinism (#35710) · f7da9cdf
Andreas Karatzas authored Mar 03, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
```
f7da9cdf
[Bugfix] Fix coord_socket assertion in DPEngineCoreProc for offline DP mode (#35916) · f22ff295
Jaewon authored Mar 03, 2026
```
Signed-off-by: Jaewon Lee <jaewon@meta.com>
```
f22ff295

03 Mar, 2026 16 commits
- [Core] Move save_tensorized_model logic to Worker (#35825) · d15c3b90
  Nick Hill authored Mar 03, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  d15c3b90
- [Model Runner V2] support dp & ep for spec decoding (#35294) · 97286a20
  zhrrr authored Mar 04, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  97286a20
- [CI/Build] Allow mounting AWS credentials for sccache S3 auth (#35912) · 12b38c0f
  Amr Mahdi authored Mar 03, 2026
```
Signed-off-by: Amr Mahdi <amrmahdi@meta.com>
```
  12b38c0f
- [Model Runner V2] Fix inputs_embeds=None bug for MM models (#35917) · 467886a0
  Woosuk Kwon authored Mar 03, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  467886a0
- [Bugfix] Fix misnamed parameter in compressed_tensors_moe.py (#35813) · a9b8b13e
  bnellnm authored Mar 03, 2026
```
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
```
  a9b8b13e
- [ROCm][CI] Fix TP size issue for `test_gpt_oss` (#35887) · e7213003
  Micah Williamson authored Mar 03, 2026
```
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
```
  e7213003
- [ROCm][Bugfix]: Disable AITER Triton ROPE by default (#35601) · 3a8eef58
  Rohan Potdar authored Mar 03, 2026
```
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
```
  3a8eef58
- [MoE Refactor] Create MK for TRTLLM Kernels (#32564) · 97995f63
  Robert Shaw authored Mar 03, 2026
```
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
```
  97995f63
- [CI] Temporarily Disable Llama4 MoE Refactor Test (#35870) · 881a6b01
  Robert Shaw authored Mar 03, 2026
```
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
```
  881a6b01
- [CI] Bump `num_speculative_tokens` to 3 in nightly DeepSeek tests (#35882) · 8e1fd5ba
  Matthew Bonanni authored Mar 03, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  8e1fd5ba
- fix: Ensure invalid audio files return 400 error (#34715) · ae88468b
  JasonCohere authored Mar 03, 2026
```
Signed-off-by: Jason Ozuzu <jasonozuzu@cohere.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
```
  ae88468b
- TRTLLM gen-full attn Test Coverage (#34986) · e05cb3b9
  ojhaanshika authored Mar 03, 2026
```
Signed-off-by: Anshika Ojha <anshikao@nvidia.com>
Co-authored-by: Anshika Ojha <anshikao@gb-nvl-059-compute09.nvidia.com>
```
  e05cb3b9
- [BugFix] Add support for MTP num_speculative_tokens > 1 with sparse MLA (#34552) · 28ef9ba3
  Lucas Wilkinson authored Mar 03, 2026
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
```
  28ef9ba3
- [ROCm] [CI] Add new fusion test cases that are relevant to vLLM IR Ops (#34307) · fb7fdc49
  TJian authored Mar 03, 2026
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  fb7fdc49
- [Frontend][1/n] Improve pooling entrypoints | classify. (#35604) · ea463978
  wang.yuqi authored Mar 03, 2026
```
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  ea463978
- [Bugfix] Avoid src/dst as None in irecv/isend_tensor_dict (#35754) · 440f0e7d
  Li, Jiang authored Mar 03, 2026
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  440f0e7d