Commits · 6c1cfbad325067c4afa12c87992f45a58ce0614b · OpenDAS / vllm_cscc

"deploy/operator/config/rbac/auth_proxy_role.yaml" did not exist on "14ce7e03bedde2b7632fa526a01a187f4b374996"

17 Mar, 2026 3 commits
- Support non-contiguous KV cache in TRTLLM fp8 dequant kernel (#36867) · 6c1cfbad
  Vadim Gimpelson authored Mar 17, 2026
```
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: Pavani Majety <pavanimajety@gmail.com>
```
  6c1cfbad
- [BugFix] Correct max memory usage for multiple KV-cache groups (#36030) · 45f526d6
  Harry Huang authored Mar 17, 2026
```
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
```
  45f526d6
- Fix some Mistral parser issues (#37209) · 5db91f0a
  Julien Denize authored Mar 17, 2026
```
Signed-off-by: juliendenize <julien.denize@mistral.ai>
```
  5db91f0a
16 Mar, 2026 37 commits
- [Feature][Frontend] add support for Cohere Embed v2 API (#37074) · 061980c3
  Walter Beller-Morales authored Mar 16, 2026
```
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
```
  061980c3
- [Custom Ops] Add functional + out variant for scaled_fp4_quant (#34389) · 3e6a1e16
  Terry Gao authored Mar 16, 2026
```
Signed-off-by: tianrengao <terrygao87@gmail.com>
```
  3e6a1e16
- Fix EagleMistralLarge3Model initialization (#37232) · 7961486a
  Julien Denize authored Mar 16, 2026
```
Signed-off-by: juliendenize <julien.denize@mistral.ai>
```
  7961486a
- [Quant][Feature] Support online MXFP8 quantization for MoE and dense models (#35448) · e5b80760
  EdalatiAli authored Mar 16, 2026
```
Signed-off-by: EdalatiAli <aliedalati@cohere.com>
```
  e5b80760
- Fix eplb nvfp4 experts hook (#37217) · fd4d9630
  Elvir Crnčević authored Mar 16, 2026
```
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Signed-off-by: Elvir Crncevic <elvir@anthropic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
```
  fd4d9630
- [compile] Enable mega aot artifact for torch 2.12+. (#37198) · e6ae4b1b
  Zhengxu Chen authored Mar 16, 2026
```
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
```
  e6ae4b1b
- [Bugfix][MultiConnector] Fix MultiConnector for SupportsHMA sub-connectors (#36549) · 2dccb38f
  zhanqiuhu authored Mar 16, 2026
  
  2dccb38f
- [BUGFIX][Mamba] Use uint64 for address in KVBlockZeroer (#37197) · d1572160
  Kunshang Ji authored Mar 17, 2026
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  d1572160
- [Misc] Add `float16` to `CacheDType` (#37199) · 93f3c8e5
  Matthew Bonanni authored Mar 16, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  93f3c8e5
- [Refactor] Consolidate GPT-OSS reasoning parser tests (#36915) · dfa8852d
  Flora Feng authored Mar 16, 2026
```
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  dfa8852d
- [torch.compile][BE] Modify cudagraph callable to check for is_forward_context_set (#36288) · 714c6e0e
  Lucas Kabela authored Mar 16, 2026
```
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
```
  714c6e0e
- [Bugfix] Fix render server crash for quantized models on CPU-only hosts (#37215) · 0fefd00e
  Sage authored Mar 16, 2026
```
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
```
  0fefd00e
- [PD][Nixl] Add support for hybrid SSM-FA models (#36687) · f5c081d4
  Nicolò Lucchesi authored Mar 16, 2026
  
  f5c081d4
- [MTP][Sparse MLA] Take advantage of native MTP support in indexer when possible (#36982) · c88ea833
  Matthew Bonanni authored Mar 16, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  c88ea833
- Add simple granite4 tool parser (#36827) · 9f9ecff4
  Max de Bayser authored Mar 16, 2026
```
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  9f9ecff4
- [Bugfix] Disable cross-layer KV cache for MLA attention backends (#37090) · ca1954d5
  haosdent authored Mar 17, 2026
```
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
```
  ca1954d5
- [Bugfix] Add error handling for FINISHED_ERROR in OpenAIServing (#37148) · 6682c231
  Chauncey authored Mar 17, 2026
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  6682c231
- [Bugfix] Relax TRTLLM KV cache contiguity assertion for cross-layer layout (#34158) · 5ae685c1
  Itay Etelis authored Mar 16, 2026
```
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
```
  5ae685c1
- [Bugfix] Add safety check and fallback for null scaling factor (#36106) · 8d8855fd
  Yuanheng Zhao authored Mar 16, 2026
```
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8d8855fd
- [Bugfix] accept redacted thinking blocks in Anthropic messages (#36992) · 0e5a9382
  Benjamin Bartels authored Mar 16, 2026
```
Signed-off-by: Benjamin Bartels <benjaminba@tiglab-ubuntu.ilab.local>
Signed-off-by: bbartels <benjamin@bartels.dev>
Co-authored-by: Benjamin Bartels <benjaminba@tiglab-ubuntu.ilab.local>
```
  0e5a9382
- [Spec Decode] Update extract_hidden_states to use deferred kv_connector clear (#37013) · 04bf5a35
  Fynn Schmitt-Ulms authored Mar 16, 2026
  
  04bf5a35
- Remove unused EVS functions in qwen3_vl.py (#37183) · 43a73f85
  Tianyu Guo authored Mar 16, 2026
```
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
```
  43a73f85
- Patch Mistral config (#37104) · ffbc2e5b
  Julien Denize authored Mar 16, 2026
```
Signed-off-by: juliendenize <julien.denize@mistral.ai>
```
  ffbc2e5b
- [Models][Qwen3 ViT] Keep `max_seqlen` on CPU to prevent D2H sync (#37139) · f9e6db30
  Lukas Geiger authored Mar 16, 2026
```
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  f9e6db30
- [Performance] Add prefetch for checkpoints to OS page cache (#36012) · f5e59ee7
  Artem Perevedentsev authored Mar 16, 2026
```
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
```
  f5e59ee7
- GLM4 tool parser: fix streaming mode (#35208) · bf9a1853
  Robin Nabel authored Mar 16, 2026
```
Signed-off-by: Robin Nabel <opensource@nabel.co>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
```
  bf9a1853
- Fix text only inputs for MRoPE models with the Transformers modelling backend (#37055) · ad041c79
  Harry Mellor authored Mar 16, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  ad041c79
- [Hardware] Replace memory related torch.cuda APIs (#37031) · 747b0681
  Kunshang Ji authored Mar 16, 2026
```
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
```
  747b0681
- Fix pipeline parallel with multimodal models with the Transformers modelling backend (#37057) · 122f75d9
  Harry Mellor authored Mar 16, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  122f75d9
- [Frontend][Misc] Remove unused log in `/is_sleeping` (#37093) · 0115e957
  Roy Wang authored Mar 16, 2026
```
Signed-off-by: esmeetu <jasonailu87@gmail.com>
```
  0115e957
- [Bugfix] Fix GDN attention crash with mixed decode/spec-decode batches (#34871) · 116ed130
  haosdent authored Mar 16, 2026
```
Signed-off-by: haosdent <haosdent@gmail.com>
```
  116ed130
- [FlashInfer] Revert block_size 16 + head_size 256 workaround on Blackwell (#36987) · 8374387b
  Vadim Gimpelson authored Mar 16, 2026
```
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
```
  8374387b
- [Bugfix] Fix Qwen2.5-Omni/Qwen3-Omni use_audio_in_video with multi-video inputs (#37147) · 912fbe95
  Isotr0py authored Mar 16, 2026
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  912fbe95
- use skip_all_guards_unsafe to drop global_state and torch_function_mode_stack... · 52131f88
  Laith Sakka authored Mar 16, 2026
```
use skip_all_guards_unsafe to drop global_state and torch_function_mode_stack guards instead of previous hacks (#36204)
Signed-off-by: Laith Sakka <lsakka@meta.com>
```
  52131f88
- [Performance][Model Loader] Skip non-local expert weights during EP model loading (#37136) · 821eb80c
  Roy Wang authored Mar 16, 2026
```
Signed-off-by: esmeetu <jasonailu87@gmail.com>
```
  821eb80c
- [ROCm] Fix KV copy methods and auto-select attention backend for ROCm (#36845) · 911355e2
  Andreas Karatzas authored Mar 16, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
```
  911355e2
- [Bugfix] fix Qwen3.5 tool calling bug (#36774) · 8d3f8f48
  Chauncey authored Mar 16, 2026
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  8d3f8f48