Commits · d56e95223917ba21af2dd87a335dcb5c0b347bfe · OpenDAS / vllm_cscc

"csrc/quantization/marlin/marlin.cuh" did not exist on "df29793dc73a83f3c86c19de967adffda1a28a93"

05 Apr, 2026 7 commits

nano_nemotron_vl: fix tensor device mismatch exception when video profiling (#39029) · d56e9522
Netanel Haber authored Apr 06, 2026
```
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
```
d56e9522

[Bug] Fix Import paths for `encoder_cudagraph` modules (#38997) · 4dd49b06

Greg Pereira authored Apr 05, 2026

Signed-off-by: greg pereira <grpereir@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

4dd49b06

[Bugfix] Fix invalid JSON in Gemma 4 streaming tool calls by stripping partial delimiters (#38992) · f53fa26e
Greg Pereira authored Apr 05, 2026
```
Signed-off-by: greg pereira <grpereir@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
```
f53fa26e

[Perf] Change Trtllm fp8 MoE to use Shuffled Weights and BlockMajorK Layout (#38993) · 1af6f78a

Wei Zhao authored Apr 05, 2026


Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

1af6f78a

[Bugfix][MoE] Fix 6-8% decode regression: prefer multi-stream shared expert overlap (#38990) · 228023b3

Martin Vit authored Apr 05, 2026


Signed-off-by: Martin Vit <martin@voipmonitor.org>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

228023b3

[Bugfix][Spec Decode] Fix extract_hidden_states for VLM models (#38987) · 9a528260
Aaron Batilo authored Apr 05, 2026
```
Signed-off-by: Aaron Batilo <abatilo@coreweave.com>
```
9a528260
[Quantization][Deprecation] Remove Petit NVFP4 (#32694) · 968ed02a
Robert Shaw authored Apr 04, 2026
```
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
```
968ed02a

04 Apr, 2026 8 commits
- Revert "[vLLM IR] gemma_rms_norm" (#38998) · 7d266abb
  Robert Shaw authored Apr 04, 2026
  
  7d266abb
- [vLLM IR] gemma_rms_norm (#38780) · 156405d2
  Xiaoshuang Wang authored Apr 05, 2026
```
Signed-off-by: Icey <1790571317@qq.com>
```
  156405d2
- [Perf][GDN] Align TMA usage with upstream FLA (#38981) · 99e5539a
  Artem Perevedentsev authored Apr 04, 2026
```
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  99e5539a
- [IR][RmsNorm] pass None if not has_weight (#38961) · a88ce94b
  Linkun authored Apr 04, 2026
```
Signed-off-by: Linkun Chen <github@lkchen.net>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  a88ce94b
- [Bugfix][CPU] Fix macOS compatibility broken by #36487 (#38970) · 2a36d8fb
  Ziming Qi authored Apr 04, 2026
```
Signed-off-by: Ziming (2imi9) <148090931+2imi9@users.noreply.github.com>
```
  2a36d8fb
- Refactor Arctic loading to use AutoWeightsLoader (#38955) · 93726b2a
  lalit10 authored Apr 03, 2026
```
Signed-off-by: Lalit Laxminarayan Bangad <lalitbangad@gmail.com>
Co-authored-by: Lalit Laxminarayan Bangad <lalitbangad@meta.com>
```
  93726b2a
- [Bugfix] Fix DSV32 weight loading (#38870) · 8617f867
  Yongye Zhu authored Apr 03, 2026
```
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
```
  8617f867
- [Bug] Fix workspace manager `_current_workspaces` size (#38853) · cab4064c
  Wentao Ye authored Apr 03, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  cab4064c
03 Apr, 2026 25 commits
- [Bugfix][LoRA] Fix missing in_proj_z in Qwen3_5ForConditionalGenerati… (#38927) · 81994e1d
  elenalil-aws authored Apr 03, 2026
```
Signed-off-by: elenalil-aws <elenalil@amazon.com>
```
  81994e1d
- [Model Runner V2] Add config validation for not-yet-supported features (#38758) · 5f1de2b1
  Nick Hill authored Apr 03, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  5f1de2b1
- [Bugfix] Re-enable Renormalize routing for TRT-LLM MoE experts (#38859) · a5a623d9
  yzong-rh authored Apr 03, 2026
```
Signed-off-by: Yifan Zong <yzong@redhat.com>
```
  a5a623d9
- [vLLM IR] add `import_ir_kernels()` to support OOT platforms (#38807) · f8c3af2d
  Xiaoshuang Wang authored Apr 04, 2026
```
Signed-off-by: Icey <1790571317@qq.com>
```
  f8c3af2d
- Fix invalid logprobs with MTP enabled and sync scheduling (#38711) · 50cd5674
  danisereb authored Apr 03, 2026
```
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
```
  50cd5674
- [Frontend] new online quantization frontend (#38138) · 7b1a7423
  Vasiliy Kuznetsov authored Apr 03, 2026
```
Signed-off-by: Vasiliy Kuznetsov <vasiliy@meta.com>
```
  7b1a7423
- [KVConnector] Skip `register_kv_caches` on profiling (#38558) · 97f92c6b
  Nicolò Lucchesi authored Apr 03, 2026
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  97f92c6b
- [Bugfix] Fix AWQ models batch invariance issues (#38670) · 46f02e00
  Yusuf Mohammad authored Apr 03, 2026
```
Signed-off-by: yusuf <yusuf@deeplearningmachine.mynet>
Signed-off-by: <>
Co-authored-by: yusuf <yusuf@deeplearningmachine.mynet>
```
  46f02e00
- [XPU] bump up xpu-kernel v0.1.5, transpose moe weights (#38342) · 6b487224
  Qiming Zhang authored Apr 03, 2026
```
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
Signed-off-by: Qiming Zhang <qiming1.zhang@intel.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
```
  6b487224
- [GDN] Eliminate GPU->CPU sync in prepare_chunk_indices during prefill (#38361) · cb10b7e8
  Artem Perevedentsev authored Apr 03, 2026
```
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
```
  cb10b7e8
- [Intel][Triton] Support `round_int8` for Intel backend (#38825) · bf8b022e
  Mieszko Dziadowiec authored Apr 03, 2026
```
Signed-off-by: Mieszko Dziadowiec <mdziadowiec@habana.ai>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
```
  bf8b022e
- [ROCm] Fix aiter persistent mode mla with q/o nhead<16 for kimi-k2.5 tp8 (#38615) · 1b117cb0
  wufann authored Apr 03, 2026
```
Signed-off-by: wufann <36477220+wufann@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  1b117cb0
- [CPU] Replace OMP initialization (#36487) · abebd932
  Anton Ivanov authored Apr 03, 2026
```
Signed-off-by: Anton Ivanov <anton.ivanov@cambridgegreys.com>
```
  abebd932
- [Frontend] feat: add streaming support for token generation endpoint (#37171) · 25f2b553
  Hyeonki Hong authored Apr 03, 2026
```
Signed-off-by: Hyeonki Hong <hyeonki.hong@moreh.io>
```
  25f2b553
- Fix Nano Nemotron VL regressions (#38655) · fa9e6802
  Netanel Haber authored Apr 03, 2026
```
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
```
  fa9e6802
- [Misc] Clean up Gemma4 implementation (#38872) · 55064354
  Isotr0py authored Apr 03, 2026
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  55064354
- [MRV2][KVConnector] Fix missing build_connector_worker_meta (#38698) · 311c9816
  Yifan Qiao authored Apr 02, 2026
```
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
```
  311c9816
- [Bug] Add e_score_correction_bias to SKIP_TENSORS (#38746) · 4729b908
  Aaron Hao authored Apr 02, 2026
```
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
```
  4729b908
- full cudagraph for flex-attn (#36298) · 8b141ed8
  shunting314 authored Apr 02, 2026
```
Signed-off-by: shunting314 <shunting@meta.com>
```
  8b141ed8
- [Model] Add Phi4ForCausalLMV for microsoft/Phi-4-reasoning-vision-15B (#38306) · 2ad7c033
  Varun Sundar Rabindranath authored Apr 03, 2026
```
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
  2ad7c033
- [ROCm][Quantization][1/N] Refactor quark_moe w_mxfp4 w/ oracle (#38774) · 103f0de5
  Bowen Bao authored Apr 02, 2026
```
Signed-off-by: Bowen Bao <bowenbao@amd.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
```
  103f0de5
- [Perf] Batch KV cache swap copies via cuMemcpyBatchAsync (#38460) · 4a06e124
  Itay Etelis authored Apr 03, 2026
```
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
```
  4a06e124
- [Kernel] Fuse FP8 output quantization into merge_attn_states (#36518) · 3bc2734d
  Carl Y authored Apr 02, 2026
```
Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com>
```
  3bc2734d
- [mla] Support fused FP8/NVFP4 output quantization in MLA attention (#35792) (#36205) · 1f5ec288
  Carl Y authored Apr 02, 2026
```
Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com>
Signed-off-by: Carl Y <4531192+carlyou@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
```
  1f5ec288
- [XPU] Initial support for GDN attention on Qwen3-next/Qwen3.5 (#33657) · ee3cf457
  Yan Ma authored Apr 03, 2026
```
Signed-off-by: Yan Ma <yan.ma@intel.com>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
```
  ee3cf457