Commits · ca21483bf900b269dd1876352ee335ab62df2ebb · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "0801ee7dccd8eed9f6fb674cdcce24b2e0ccb46e"

19 Mar, 2026 7 commits
- [MISC] fix pin_memory=torch.cuda.is_available(), use is_pin_memory_available (#37415) · ca21483b
  Kunshang Ji authored Mar 19, 2026
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  ca21483b
- Support temporal compression for Nemotron-3-VL videos (#36808) · 0b6d5262
  Collin McCarthy authored Mar 19, 2026
```
Signed-off-by: Collin McCarthy <cmccarthy@nvidia.com>
```
  0b6d5262
- [Perf] Fix slow hasattr in CUDAGraphWrapper.__getattr__ (#37425) · d3cc3795
  Ziming Huang authored Mar 19, 2026
```
Signed-off-by: 智鸣 <hzm414167@alibaba-inc.com>
```
  d3cc3795
- [Perf] Optimize token_embed for pooling models, 1.0% token throughput improvement (#37347) · e37ff5b5
  Wentao Ye authored Mar 18, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  e37ff5b5
- [bug] Fix deadlock with pause resume and collective_rpc (#37024) · 6accb21f
  Aaron Hao authored Mar 18, 2026
```
Signed-off-by: hao-aaron <ahao@anyscale.com>
```
  6accb21f
- [Model Runner V2] Spec decode rejection sampler logprobs support (#37237) · 053f3b63
  Giancarlo Delfin authored Mar 18, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  053f3b63
- [Bugfix] Zero-init MLA attention output buffers to prevent NaN from CUDA graph padding (#37442) · ef2c4f77
  Elvir Crnčević authored Mar 19, 2026
```
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
```
  ef2c4f77
18 Mar, 2026 12 commits
- [Bugfix] Expand quantization method support in perf metrics (#37231) · 828f862a
  Thillai Chithambaram authored Mar 18, 2026
```
Signed-off-by: Thillai Chithambaram <thillaichithambaram.a@gmail.com>
```
  828f862a
- [Bugfix] Fix KV scales inconsistency in fp8 MLA & FlashInfer kv_cache_dtype... · 577df69b
  Andy Lo authored Mar 18, 2026
```
[Bugfix] Fix KV scales inconsistency in fp8 MLA & FlashInfer kv_cache_dtype "auto" leading to gibberish (#37054)
Signed-off-by: Andy Lo <andy@mistral.ai>
```
  577df69b
- [Model Runner V2] Spec decode rejection sampler greedy support (#37238) · 04244fd0
  Giancarlo Delfin authored Mar 18, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  04244fd0
- fix(worker): optimize swap_states to copy only active token prefixes (#34733) · 00910171
  Philip Ottesen authored Mar 18, 2026
```
Signed-off-by: Philip Ottesen <phiott256@gmail.com>
```
  00910171
- [V0 Deprecation] Deprecate virtual engine (#37195) · 0d81a1fe
  Wentao Ye authored Mar 18, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  0d81a1fe
- [bugfix][async scheduling] fix extra cuda context in device 0 with EP/DP (#37449) · 70b81c4f
  youkaichao authored Mar 19, 2026
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  70b81c4f
- [kv_offload+HMA][2/N]: Support multiple KV groups in GPULoadStoreSpec (#36642) · 5dd8df07
  Or Ozeri authored Mar 18, 2026
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  5dd8df07
- [Log] Reduce duplicate log (#37313) · c373b5c0
  Wentao Ye authored Mar 18, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  c373b5c0
- elastic_ep: Fix stateless group port races (#36330) · de1a86b7
  Itay Alroy authored Mar 18, 2026
```
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
```
  de1a86b7
- [kv_offload+HMA][0/N]: Support block-level preemption handling (#34805) · fcf0687b
  Or Ozeri authored Mar 18, 2026
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
```
  fcf0687b
- [Hardware][TPU] Add supports_async_scheduling() method to Executor interface... · a0dd1995
  gxd3 authored Mar 17, 2026
```
[Hardware][TPU] Add supports_async_scheduling() method to Executor interface so that it can be extended for Executor implementations. (#36924)
Signed-off-by: Guangxiang Du <gxd@google.com>
```
  a0dd1995
- [ROCm][Quantization] add fp8xfp8 attn support for rocm_aiter_unified_attn (#36927) · e6c47977
  Divakar Verma authored Mar 17, 2026
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  e6c47977
17 Mar, 2026 9 commits
- [Bugfix][ROCm] Fix worker startup OOM on ROCm by skipping unreliable cudagraph... · e8f9dbc3
  JartX authored Mar 17, 2026
```
[Bugfix][ROCm] Fix worker startup OOM on ROCm by skipping unreliable cudagraph memory profiling (#36720)
Signed-off-by: JartX <sagformas@epdcenter.es>
```
  e8f9dbc3
- Make KV connector metadata build overridable via plugin (#37336) · de35c06c
  Yong Hoon Shin authored Mar 17, 2026
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  de35c06c
- [Torch 2.11] Guard torch._C._cpu attribute checks for forward compatibility (#35673) · 68f783a7
  Andrey Talman authored Mar 17, 2026
```
Signed-off-by: atalman <atalman@fb.com>
```
  68f783a7
- [Perf] Optimize top-k search in apply_top_k_top_p_triton sampler (#37225) · 51b2333b
  Michael Goin authored Mar 17, 2026
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  51b2333b
- [Bugfix] Fix DP MTP Dummy Run (#35243) · f63ed7b5
  Benjamin Chislett authored Mar 17, 2026
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
  f63ed7b5
- [Bugfix] Fix NemotronH MTP + Chunked Prefill (#35447) · 8a680463
  Benjamin Chislett authored Mar 17, 2026
  
  8a680463
- [Bugfix] dtype mismatch in ngram gpu propose (#37246) · 17c1bdf3
  PatchyTIS authored Mar 17, 2026
```
Signed-off-by: PatchouliTaisa <patchychen@tencent.com>
Co-authored-by: PatchouliTaisa <patchychen@tencent.com>
```
  17c1bdf3
- Support non-contiguous KV cache in TRTLLM fp8 dequant kernel (#36867) · 6c1cfbad
  Vadim Gimpelson authored Mar 17, 2026
```
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: Pavani Majety <pavanimajety@gmail.com>
```
  6c1cfbad
- [BugFix] Correct max memory usage for multiple KV-cache groups (#36030) · 45f526d6
  Harry Huang authored Mar 17, 2026
```
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
```
  45f526d6
16 Mar, 2026 11 commits
- [BUGFIX][Mamba] Use uint64 for address in KVBlockZeroer (#37197) · d1572160
  Kunshang Ji authored Mar 17, 2026
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  d1572160
- [Misc] Add `float16` to `CacheDType` (#37199) · 93f3c8e5
  Matthew Bonanni authored Mar 16, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  93f3c8e5
- [MTP][Sparse MLA] Take advantage of native MTP support in indexer when possible (#36982) · c88ea833
  Matthew Bonanni authored Mar 16, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  c88ea833
- [Bugfix] Disable cross-layer KV cache for MLA attention backends (#37090) · ca1954d5
  haosdent authored Mar 17, 2026
```
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
```
  ca1954d5
- [Bugfix] Relax TRTLLM KV cache contiguity assertion for cross-layer layout (#34158) · 5ae685c1
  Itay Etelis authored Mar 16, 2026
```
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
```
  5ae685c1
- [Spec Decode] Update extract_hidden_states to use deferred kv_connector clear (#37013) · 04bf5a35
  Fynn Schmitt-Ulms authored Mar 16, 2026
  
  04bf5a35
- [Hardware] Replace memory related torch.cuda APIs (#37031) · 747b0681
  Kunshang Ji authored Mar 16, 2026
```
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
```
  747b0681
- [Bugfix] Fix GDN attention crash with mixed decode/spec-decode batches (#34871) · 116ed130
  haosdent authored Mar 16, 2026
```
Signed-off-by: haosdent <haosdent@gmail.com>
```
  116ed130
- [FlashInfer] Revert block_size 16 + head_size 256 workaround on Blackwell (#36987) · 8374387b
  Vadim Gimpelson authored Mar 16, 2026
```
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
```
  8374387b
- [Model Runner V2] Fix processed logits in sample() (#37144) · 96efb914
  Woosuk Kwon authored Mar 16, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  96efb914
- [Bugfix] Avoid LD_PRELOAD check on MacOS (#37145) · 7362b445
  Li, Jiang authored Mar 16, 2026
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  7362b445
14 Mar, 2026 1 commit
- [Bugfix] Fix xgrammar dtype mismatch on macOS CPU inference (#32384) · 821fde2d
  Karan Bansal authored Mar 14, 2026
```
Signed-off-by: Karan Bansal <karanb192@gmail.com>
Co-authored-by: Inokinoki <inoki@inoki.cc>
```
  821fde2d