Commits · f1740006e47d580656668ba5a9253a4e4340e198 · OpenDAS / vllm_cscc

"fern/pages/kubernetes/autoscaling.md" did not exist on "09f2314df031aab007f1fde8506966f34ae0c6fa"

18 Mar, 2026 1 commit
- [ROCm][Quantization] add fp8xfp8 attn support for rocm_aiter_unified_attn (#36927) · e6c47977
  Divakar Verma authored Mar 17, 2026
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  e6c47977
17 Mar, 2026 9 commits
- [Bugfix][ROCm] Fix worker startup OOM on ROCm by skipping unreliable cudagraph... · e8f9dbc3
  JartX authored Mar 17, 2026
```
[Bugfix][ROCm] Fix worker startup OOM on ROCm by skipping unreliable cudagraph memory profiling (#36720)
Signed-off-by: JartX <sagformas@epdcenter.es>
```
  e8f9dbc3
- Make KV connector metadata build overridable via plugin (#37336) · de35c06c
  Yong Hoon Shin authored Mar 17, 2026
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  de35c06c
- [Torch 2.11] Guard torch._C._cpu attribute checks for forward compatibility (#35673) · 68f783a7
  Andrey Talman authored Mar 17, 2026
```
Signed-off-by: atalman <atalman@fb.com>
```
  68f783a7
- [Perf] Optimize top-k search in apply_top_k_top_p_triton sampler (#37225) · 51b2333b
  Michael Goin authored Mar 17, 2026
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  51b2333b
- [Bugfix] Fix DP MTP Dummy Run (#35243) · f63ed7b5
  Benjamin Chislett authored Mar 17, 2026
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
  f63ed7b5
- [Bugfix] Fix NemotronH MTP + Chunked Prefill (#35447) · 8a680463
  Benjamin Chislett authored Mar 17, 2026
  
  8a680463
- [Bugfix] dtype mismatch in ngram gpu propose (#37246) · 17c1bdf3
  PatchyTIS authored Mar 17, 2026
```
Signed-off-by: PatchouliTaisa <patchychen@tencent.com>
Co-authored-by: PatchouliTaisa <patchychen@tencent.com>
```
  17c1bdf3
- Support non-contiguous KV cache in TRTLLM fp8 dequant kernel (#36867) · 6c1cfbad
  Vadim Gimpelson authored Mar 17, 2026
```
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: Pavani Majety <pavanimajety@gmail.com>
```
  6c1cfbad
- [BugFix] Correct max memory usage for multiple KV-cache groups (#36030) · 45f526d6
  Harry Huang authored Mar 17, 2026
```
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
```
  45f526d6
16 Mar, 2026 11 commits
- [BUGFIX][Mamba] Use uint64 for address in KVBlockZeroer (#37197) · d1572160
  Kunshang Ji authored Mar 17, 2026
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  d1572160
- [Misc] Add `float16` to `CacheDType` (#37199) · 93f3c8e5
  Matthew Bonanni authored Mar 16, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  93f3c8e5
- [MTP][Sparse MLA] Take advantage of native MTP support in indexer when possible (#36982) · c88ea833
  Matthew Bonanni authored Mar 16, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  c88ea833
- [Bugfix] Disable cross-layer KV cache for MLA attention backends (#37090) · ca1954d5
  haosdent authored Mar 17, 2026
```
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
```
  ca1954d5
- [Bugfix] Relax TRTLLM KV cache contiguity assertion for cross-layer layout (#34158) · 5ae685c1
  Itay Etelis authored Mar 16, 2026
```
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
```
  5ae685c1
- [Spec Decode] Update extract_hidden_states to use deferred kv_connector clear (#37013) · 04bf5a35
  Fynn Schmitt-Ulms authored Mar 16, 2026
  
  04bf5a35
- [Hardware] Replace memory related torch.cuda APIs (#37031) · 747b0681
  Kunshang Ji authored Mar 16, 2026
```
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
```
  747b0681
- [Bugfix] Fix GDN attention crash with mixed decode/spec-decode batches (#34871) · 116ed130
  haosdent authored Mar 16, 2026
```
Signed-off-by: haosdent <haosdent@gmail.com>
```
  116ed130
- [FlashInfer] Revert block_size 16 + head_size 256 workaround on Blackwell (#36987) · 8374387b
  Vadim Gimpelson authored Mar 16, 2026
```
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
```
  8374387b
- [Model Runner V2] Fix processed logits in sample() (#37144) · 96efb914
  Woosuk Kwon authored Mar 16, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  96efb914
- [Bugfix] Avoid LD_PRELOAD check on MacOS (#37145) · 7362b445
  Li, Jiang authored Mar 16, 2026
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  7362b445
14 Mar, 2026 5 commits
- [Bugfix] Fix xgrammar dtype mismatch on macOS CPU inference (#32384) · 821fde2d
  Karan Bansal authored Mar 14, 2026
```
Signed-off-by: Karan Bansal <karanb192@gmail.com>
Co-authored-by: Inokinoki <inoki@inoki.cc>
```
  821fde2d
- [Model Runner V2] Add Support for XD-RoPE (#36817) · 3ed46f37
  Santino Ramos authored Mar 14, 2026
```
Signed-off-by: Santino Ramos <elsantinoramos@gmail.com>
```
  3ed46f37
- [Bug] Fix Failure in /v1/chat/completions/render for Multimodal Requests... · 4a718e77
  Sergey Zinchenko authored Mar 14, 2026
```
[Bug] Fix Failure in /v1/chat/completions/render for Multimodal Requests (https://github.com/vllm-project/vllm/issues/35665) (#35684)
```
  4a718e77
- [V1] Remove pin_memory() in async_copy_to_gpu to fix sporadic stalls (#37006) · a116f969
  sbeurnier authored Mar 14, 2026
```
Signed-off-by: Sebastien Beurnier <sbeurnier@together.ai>
```
  a116f969
- [UX] Improve UX of CPU backend (#36968) · 092ace9e
  Li, Jiang authored Mar 14, 2026
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: Li, Jiang <bigpyj64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  092ace9e
13 Mar, 2026 10 commits
- [Feat][Bugfix] Enable additional dimension for Flashinfer MLA and fix routing dtype (#36931) · 367cf5cd
  Dimitrios Bariamis authored Mar 14, 2026
```
Signed-off-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
Co-authored-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
```
  367cf5cd
- [Refactor] Consolidate SupportsEagle (#36063) · 8b346309
  Benjamin Chislett authored Mar 13, 2026
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
  8b346309
- [BugFix] Fix "DP Coordinator receives unexpected..." messages (#37008) · 54a6db82
  Nick Hill authored Mar 13, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  54a6db82
- [Bugfix][Spec Decode] Avoid double call of Ngram CPU (#36952) · d0b40297
  Ekagra Ranjan authored Mar 13, 2026
```
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
```
  d0b40297
- [Frontend][Core] Re-add shutdown timeout - allowing in-flight requests to finish (#36666) · 7afe0faa
  Mark McLoughlin authored Mar 13, 2026
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
```
  7afe0faa
- [2/N] Elastic EP Milestone 2: Integrating NIXL-EP (#35627) · d5af196c
  Itay Alroy authored Mar 13, 2026
```
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Yongji Wu <wuyongji317@gmail.com>
Co-authored-by: Ron Tourgeman <rtourgeman@nvidia.com>
```
  d5af196c
- [kv_offload+HMA][1/N]: Support multiple KV groups in OffloadingSpec (#36610) · cfaf4668
  Or Ozeri authored Mar 13, 2026
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  cfaf4668
- Enable RoPE+KV cache fusion for ROCm AITER FA (non-shuffle layout) (#35786) · a4ad9db5
  Rohan Potdar authored Mar 13, 2026
```
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
```
  a4ad9db5
- fix(kv-cache): increase hybrid attention grouping threshold from 1.25 to 1.5 (#36684) · 891c60dc
  jaime campos salas authored Mar 12, 2026
```
Signed-off-by: Jaime Campos Salas <jaime.campos.salas@gmail.com>
```
  891c60dc
- [Model Runner V2] Some code simplification (#36929) · cd32d6f5
  Nick Hill authored Mar 12, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  cd32d6f5
12 Mar, 2026 4 commits
- [Attention] Use FA4 for MLA prefill (#34732) · f444c05c
  Matthew Bonanni authored Mar 12, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  f444c05c
- [Kernel] Add FP8 KV cache support to Triton MLA decode attention (#34597) · a1257fd1
  grimulkan authored Mar 12, 2026
```
Signed-off-by: grimulkan <grimulkan@gmail.com>
```
  a1257fd1
- [Hardware] Replace torch.cuda.device_count/current_device/set_device API (#36145) · 53ec16a7
  Kunshang Ji authored Mar 12, 2026
```
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  53ec16a7
- [BugFix] Fix multiple/duplicate stdout prefixes (#36822) · 36735fd7
  Nick Hill authored Mar 11, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  36735fd7