Commits · f85e479e66167532feddc9aafd56562402def9d9 · OpenDAS / vllm_cscc

"examples/backends/vllm/launch/agg_multimodal_llama.sh" did not exist on "22d910a5cdd557e9457b81b7084cded9d7f93604"

23 Mar, 2026 1 commit
- [Feature] ViT Full CUDA Graph (#35963) · f85e479e
  Baorun (Lauren) Mu authored Mar 23, 2026
```
Signed-off-by: Baorun Mu <bmu@nvidia.com>
```
  f85e479e
22 Mar, 2026 5 commits
- [Model Runner V2] Enable piecewise CUDA graphs for pipeline parallelism (#35162) · 63f49b8b
  zhanqiuhu authored Mar 22, 2026
```
Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
```
  63f49b8b
- [MRV2] Use FP64 for Gumbel noise (#37798) · a5e9d511
  Woosuk Kwon authored Mar 22, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  a5e9d511
- [MRV2] Skip hidden states allocation for PW CUDA graphs (#37818) · ce9b1d76
  Woosuk Kwon authored Mar 22, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  ce9b1d76
- [Model Runner V2] Support multi-modal embeddings for spec decode model (#36097) · b3e84601
  Giancarlo Delfin authored Mar 22, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
```
  b3e84601
- [Bugfix] Fix pooling non-determinism from pinned prompt_lens aliasing (#37775) · 66f927f2
  Andreas Karatzas authored Mar 21, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
```
  66f927f2
21 Mar, 2026 2 commits

Add tensor IPC transfer mechanism for multimodal data (#32104) · 80b70884

Brandon Pelfrey authored Mar 21, 2026


Signed-off-by: Brandon Pelfrey <bpelfrey@nvidia.com>
Signed-off-by: Brandon Pelfrey <brandonpelfrey@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>

80b70884

[Hybrid] calling get_mamba_groups() once at MambaCopyBuffers.create() (#37318) · 298e5108
Francesco Fusco authored Mar 21, 2026
```
Signed-off-by: Francesco Fusco <ffu@zurich.ibm.com>
```
298e5108

20 Mar, 2026 13 commits
- elastic_ep: Fix issues with repeated scale up/down cycles (#37131) · c57d38d6
  Itay Alroy authored Mar 21, 2026
```
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Ron Tourgeman <rtourgeman@nvidia.com>
```
  c57d38d6
- [BugFix] Allow qk_nope_head_dim=192 in FlashInfer MLA backend checks (#37475) · e5ed6c6c
  Kaihang Jiang authored Mar 20, 2026
```
Signed-off-by: Kaihang Jiang <kaihangj@nvidia.com>
```
  e5ed6c6c
- [Model Runner V2] Support Streaming Inputs (#37028) · 85f671b8
  Santino Ramos authored Mar 20, 2026
```
Signed-off-by: Santino Ramos <elsantinoramos@gmail.com>
```
  85f671b8
- [Attention] Support distinguishing between short extends and decodes (#37303) · e1d85e5c
  Lucas Wilkinson authored Mar 20, 2026
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  e1d85e5c
- fix CUDAGraph memory being counted twice (#37426) · 79eb9369
  Peter Pan authored Mar 21, 2026
```
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
Signed-off-by: Peter Pan <peter.pan@daocloud.io>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
```
  79eb9369
- [MRV2] Avoid recompilation of _gather_block_tables_kernel (#37645) · e80cfe57
  Woosuk Kwon authored Mar 20, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  e80cfe57
- [Metrics] Some small refactoring for better maintainability (#33898) · 880be2b1
  Martin Hickey authored Mar 20, 2026
```
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
```
  880be2b1
- [Model] Deprecate the score task (this will not affect users). (#37537) · ed359c49
  wang.yuqi authored Mar 20, 2026
```
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
```
  ed359c49
- [Model Runner V2] Fix draft logits not populated during cudagraph replay (#37639) · dcee9be9
  Giancarlo Delfin authored Mar 20, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  dcee9be9
- [Model Runner V2] fix draft attention metadata generation (#37364) · 39474513
  Giancarlo Delfin authored Mar 19, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  39474513
- fix: disambiguate multimodal prefix cache keys (#36708) · 269bf46d
  tianshu-Michael-yu authored Mar 19, 2026
```
Signed-off-by: tianshu.yu <tianshuyu.formal@gmail.com>
```
  269bf46d
- Fix DP coordinator ZMQ port TOCTOU (#37452) · ca1ac1a4
  Itay Alroy authored Mar 20, 2026
```
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
```
  ca1ac1a4
- [ROCm][Bugfix] fix cache block size mismatch for aiter unified attention (#37606) · 4ca3fa6b
  Divakar Verma authored Mar 19, 2026
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  4ca3fa6b
19 Mar, 2026 11 commits
- [Refactor] Remove dead code in pooling model (#37572) · 2be1a0f7
  Wentao Ye authored Mar 19, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  2be1a0f7
- [CPU][UX] Do not crash when tcmalloc/libiomp are not ldpreloaded (#37561) · 2890aecc
  Fadi Arafeh authored Mar 19, 2026
```
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
```
  2890aecc
- [MRV2] Use fp32 for draft logits (#37526) · 40b8363b
  Woosuk Kwon authored Mar 19, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  40b8363b
- Fix KV Offloading + MLA AssertionError by using num_kv_heads=1 in cpu… (#37536) · e390742c
  XueLiang Yang authored Mar 19, 2026
```
Signed-off-by: xueliangyang-oeuler <yxl546827391@gmail.com>
Co-authored-by: xueliangyang-oeuler <yxl546827391@gmail.com>
```
  e390742c
- [MISC] fix pin_memory=torch.cuda.is_available(), use is_pin_memory_available (#37415) · ca21483b
  Kunshang Ji authored Mar 19, 2026
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  ca21483b
- Support temporal compression for Nemotron-3-VL videos (#36808) · 0b6d5262
  Collin McCarthy authored Mar 19, 2026
```
Signed-off-by: Collin McCarthy <cmccarthy@nvidia.com>
```
  0b6d5262
- [Perf] Fix slow hasattr in CUDAGraphWrapper.__getattr__ (#37425) · d3cc3795
  Ziming Huang authored Mar 19, 2026
```
Signed-off-by: 智鸣 <hzm414167@alibaba-inc.com>
```
  d3cc3795
- [Perf] Optimize token_embed for pooling models, 1.0% token throughput improvement (#37347) · e37ff5b5
  Wentao Ye authored Mar 18, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  e37ff5b5
- [bug] Fix deadlock with pause resume and collective_rpc (#37024) · 6accb21f
  Aaron Hao authored Mar 18, 2026
```
Signed-off-by: hao-aaron <ahao@anyscale.com>
```
  6accb21f
- [Model Runner V2] Spec decode rejection sampler logprobs support (#37237) · 053f3b63
  Giancarlo Delfin authored Mar 18, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  053f3b63
- [Bugfix] Zero-init MLA attention output buffers to prevent NaN from CUDA graph padding (#37442) · ef2c4f77
  Elvir Crnčević authored Mar 19, 2026
```
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
```
  ef2c4f77
18 Mar, 2026 8 commits
- [Bugfix] Expand quantization method support in perf metrics (#37231) · 828f862a
  Thillai Chithambaram authored Mar 18, 2026
```
Signed-off-by: Thillai Chithambaram <thillaichithambaram.a@gmail.com>
```
  828f862a
- [Bugfix] Fix KV scales inconsistency in fp8 MLA & FlashInfer kv_cache_dtype... · 577df69b
  Andy Lo authored Mar 18, 2026
```
[Bugfix] Fix KV scales inconsistency in fp8 MLA & FlashInfer kv_cache_dtype "auto" leading to gibberish (#37054)
Signed-off-by: Andy Lo <andy@mistral.ai>
```
  577df69b
- [Model Runner V2] Spec decode rejection sampler greedy support (#37238) · 04244fd0
  Giancarlo Delfin authored Mar 18, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  04244fd0
- fix(worker): optimize swap_states to copy only active token prefixes (#34733) · 00910171
  Philip Ottesen authored Mar 18, 2026
```
Signed-off-by: Philip Ottesen <phiott256@gmail.com>
```
  00910171
- [V0 Deprecation] Deprecate virtual engine (#37195) · 0d81a1fe
  Wentao Ye authored Mar 18, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  0d81a1fe
- [bugfix][async scheduling] fix extra cuda context in device 0 with EP/DP (#37449) · 70b81c4f
  youkaichao authored Mar 19, 2026
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  70b81c4f
- [kv_offload+HMA][2/N]: Support multiple KV groups in GPULoadStoreSpec (#36642) · 5dd8df07
  Or Ozeri authored Mar 18, 2026
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  5dd8df07
- [Log] Reduce duplicate log (#37313) · c373b5c0
  Wentao Ye authored Mar 18, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  c373b5c0