Commits · d0cd736caadafea1ec1721737af432d8b0a7e919 · OpenDAS / vllm_cscc

"examples/backends/vllm/deploy/README.md" did not exist on "3ea3d59b914ad91a95b128c363116797b5f4669f"

10 Mar, 2026 6 commits
- [Bugfix] Fix `RuntimeError: Already borrowed` that degrades VLM serving... · d0cd736c
  hallerite authored Mar 09, 2026
```
[Bugfix] Fix `RuntimeError: Already borrowed` that degrades VLM serving throughput under concurrent load. (#36557)
Signed-off-by: hallerite <hallerite@users.noreply.github.com>
Signed-off-by: hallerite <git@hallerite.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  d0cd736c
- Remove unused disable_fallback field (#36546) · 04b67d8f
  Zhuohan Li authored Mar 09, 2026
  
  04b67d8f
- [Perf] Compute maxsim in worker side, reducing redundant copies, 2.7% E2E... · 7279374f
  Wentao Ye authored Mar 09, 2026
```
[Perf] Compute maxsim in worker side, reducing redundant copies, 2.7% E2E throughput improvement (#36159)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  7279374f
- [BugFix] Remove incorrect assert in split_decodes_and_prefills (#36553) · 006aea17
  Woosuk Kwon authored Mar 09, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  006aea17
- [Model] Add HyperCLOVAX-SEED-Think-32B vision-language model support (#31471) · 0836be3b
  Hojin Yang authored Mar 10, 2026
```
Signed-off-by: effortprogrammer <yhjhoward7@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  0836be3b
- [Bugfix] Fix Qwen3-Next in_proj_ba weight sharding with TP > 1 (#36242) · 4e95ec11
  Ajay Anubolu authored Mar 09, 2026
```
Signed-off-by: AjAnubolu <anuboluajay@gmail.com>
```
  4e95ec11
09 Mar, 2026 34 commits

[bugfix] fix nvlink for nixl/ucx (#36475) · f85b4eda
youkaichao authored Mar 10, 2026
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
f85b4eda
[Model Runner V2] Add model_state inputs to CUDA graph capture (#36544) · 2a194ddd
Woosuk Kwon authored Mar 09, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
2a194ddd

add nemotron v3 reasoning parser (#36393) · 203a7f27

Shaun Kotek authored Mar 10, 2026


Signed-off-by: Shaun Kotek - Nvidia <skotek@nvidia.com>
Co-authored-by: root <root@gpu-259.slurm-workers-slurm.slurm.svc.cluster.local>

203a7f27

[MRV2] Extensible CG dispatch rework (#35959) · 483463f7
Lucas Wilkinson authored Mar 09, 2026
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
483463f7
[MTP][Misc] Clean up dead code (#36507) · 4e571ce6
Matthew Bonanni authored Mar 09, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
4e571ce6
[BE] Rename `should_torch_compile_mm_vit` to `should_torch_compile_mm_encoder` (#36281) · 3fd03f1e
Lucas Kabela authored Mar 09, 2026
```
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
```
3fd03f1e
[Model Runner V2] Use NamedTuple for `execute_model_state` (#35930) · 10a5f4d5
Woosuk Kwon authored Mar 09, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
10a5f4d5

[Misc] Refactored 5 duplicate helper functions that were copied-pasted across... · 8d6b3d5d

Taneem Ibrahim authored Mar 09, 2026


[Misc] Refactored 5 duplicate helper functions that were copied-pasted across multiple parsers (#36436)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>

8d6b3d5d

[torch.compile] Rename `compile_ranges_split_points` to `compile_ranges_endpoints` (#36027) · 4b87ffbe

Copilot authored Mar 09, 2026

Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ProExpertProg <11367180+ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

4b87ffbe

Fix/resupport nongated fused moe triton (#36412) · fa028207

Shaun Kotek authored Mar 09, 2026


Signed-off-by: Shaun Kotek - Nvidia <skotek@nvidia.com>
Signed-off-by: Natan Bagrov <nbagrov@nvidia.com>
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Signed-off-by: liweiguang <codingpunk@gmail.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: cong-or <conchubhar.gannon@gmail.com>
Signed-off-by: Tushar Shetty <tushar.shetty@abbyy.com>
Signed-off-by: Tushar Shetty <54362365+tusharshetty61@users.noreply.github.com>
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Signed-off-by: Xin Yang <xyangx@amazon.com>
Signed-off-by: Kevin H. Luu <khluu000@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: nvnbagrov <nbagrov@nvidia.com>
Co-authored-by: Sage <80211083+sagearc@users.noreply.github.com>
Co-authored-by: danisereb <daserebrenik@nvidia.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Weiguang Li <codingpunk@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: cong-or <conchubhar.gannon@gmail.com>
Co-authored-by: Tushar Shetty <54362365+tusharshetty61@users.noreply.github.com>
Co-authored-by: liuzhenwei <zhenwei.liu@intel.com>
Co-authored-by: Xin Yang <105740670+xyang16@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

fa028207

[Model Runner V2] Add dummy profile_cudagraph_memory API (#36520) · 6e956d9e
Woosuk Kwon authored Mar 09, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
6e956d9e

[ROCm][CI] Fix ROCm attention backend validation for head sizes, block sizes,... · c174d54f

Andreas Karatzas authored Mar 09, 2026


[ROCm][CI] Fix ROCm attention backend validation for head sizes, block sizes, and compute capability checks (#36292)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>

c174d54f

[Misc] fix typo: dependant -> dependent (2 lines change) (#36511) · 55d27cca
SoluMilken authored Mar 10, 2026
```
Signed-off-by: SoluMilken <ypiheyn.imm02g@g2.nctu.edu.tw>
```
55d27cca

[Attention][Perf][Kernel] Replace torch.cat with vectorized CUDA kernel MLA... · 580864d8

Roberto L. Castro authored Mar 09, 2026


[Attention][Perf][Kernel] Replace torch.cat with vectorized CUDA kernel MLA query concat - DeepSeek-V3.2 (#34917)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>

580864d8

[ROCM] Optimize the fused_topk_bias to use aiter instead of fallback torch ops. (#36253) · 70485a11
Taoyu Zhu authored Mar 10, 2026
```
Signed-off-by: zhutaoyu <zhutaoyu97@gmail.com>
```
70485a11
[Bugfix] Clear stale CG keys after memory profiling (#36416) · 00c4cb56
Matthew Bonanni authored Mar 09, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
00c4cb56
[Refactor] Simplify `chat_completion_full_generator` for tool parsers (#35634) · 941e52c2
Wentao Ye authored Mar 09, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
941e52c2
[Bug] Fix pooling model benchmark script (#36300) · be292b7c
Wentao Ye authored Mar 09, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
be292b7c
Reapply [Attention] Refactor `check_and_update_config` (#35122) · 77a73458
Matthew Bonanni authored Mar 09, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
77a73458

Support online use_audio_in_video (#36319) · 5578f2a4

Tianyu Guo authored Mar 09, 2026


Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

5578f2a4

[Frontend] Move warmup into Renderer (#36482) · 3ec21150
Cyrus Leung authored Mar 09, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
3ec21150
[MM Encoder] Default to use TORCH_SDPA backend for ViT on Volta/Turing GPU (#36472) · b0906d8b
Isotr0py authored Mar 09, 2026
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
b0906d8b
[Deprecation][1/2] Remove items deprecated in v0.18 (#36470) · f96c3ab0
Cyrus Leung authored Mar 09, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
f96c3ab0
[Kernel] Add fused_sigmoid_gating_delta_rule_update kernel for Qwen3 Next (#35777) · dc6b5784
Xin Yang authored Mar 08, 2026
```
Signed-off-by: Xin Yang <xyangx@amazon.com>
```
dc6b5784
[Frontend] Add Support for MM Encoder/Decoder Beam Search (Online Transcriptions) (#36160) · 65a4da15
Alex Brooks authored Mar 08, 2026
```
Signed-off-by: Alex Brooks <albrooks@redhat.com>
```
65a4da15
[Bugfix] Avoid to replace non-tensor members in cpu model runner (#36430) · 217f2759
Li, Jiang authored Mar 09, 2026
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
217f2759
[Frontend][2/n] Improve pooling entrypoints | embed. (#36110) · fff3711a
wang.yuqi authored Mar 09, 2026
```
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
```
fff3711a
[Bugfix] Skip out-of-stage layers in get_layers_from_vllm_config for pipeline parallel (#36243) · c4d859c2
Tushar Shetty authored Mar 09, 2026
```
Signed-off-by: Tushar Shetty <tushar.shetty@abbyy.com>
Signed-off-by: Tushar Shetty <54362365+tusharshetty61@users.noreply.github.com>
```
c4d859c2
feat(attention): extract KV-cache update from FlexAttention backend (#36263) · 74743104
cong-or authored Mar 09, 2026
```
Signed-off-by: cong-or <conchubhar.gannon@gmail.com>
```
74743104
[Misc] Move processors to `transformers_utils` (#35953) · d62856b9
Cyrus Leung authored Mar 09, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
d62856b9
Increase Flexibility for OOV Multimodal Token Handling (#34858) · bd2659a5
Alex Brooks authored Mar 08, 2026
```
Signed-off-by: Alex Brooks <albrooks@redhat.com>
```
bd2659a5
fix: Use iterator as not to store all the file loads in memory at once (#36149) · 90512b2e
Shaun Kotek authored Mar 09, 2026
```
Signed-off-by: Shaun Kotek - Nvidia <skotek@nvidia.com>
```
90512b2e

[Examples][1/n] Resettle basic examples. (#35579) · dcf8862f

wang.yuqi authored Mar 09, 2026

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

dcf8862f

[Bugfix] Fix CPU OMP autobind assertion to use local_world_size (#35815) · 43aa3892

Weiguang Li authored Mar 09, 2026


Signed-off-by: liweiguang <codingpunk@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>

43aa3892