Commits · 38de822310239a50c610a420e2ed9f6dd670d5a7 · OpenDAS / vllm_cscc · GitLab

26 Mar, 2026 11 commits
- [Model] Add torch.compile support for InternVL vision encoder (#38049) · 38de8223
  Terry Gao authored Mar 25, 2026
```
Signed-off-by: tianrengao <terrygao87@gmail.com>
```
  38de8223
- Add `/v1/chat/completions/batch` endpoint for batched chat completions (#38011) · 29080945
  Matej Rojec authored Mar 26, 2026
```
Signed-off-by: Matej Rojec <64556640+MatejRojec@users.noreply.github.com>
```
  29080945
- [Bugfix][CI] Fix Marlin FP8 Linear Kernel for Compressed Tensors Format (#38092) · e6bf9f15
  BadrBasowid authored Mar 26, 2026
```
Signed-off-by: BadrBasowid <Badr.Basowid@gmail.com>
Signed-off-by: BadrBasowid <61441185+BadrBasowid@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  e6bf9f15
- Relocate Encoder CUDA graph manager (#38116) · 144030c8
  Woosuk Kwon authored Mar 25, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
```
  144030c8
- [Tool Parser][1/3] Pass tools to ToolParser constructor (#38029) · e2db2b42
  Flora Feng authored Mar 25, 2026
```
Signed-off-by: sfeng33 <4florafeng@gmail.com>
```
  e2db2b42
- [Revert] Remove DeepGEMM availability check in DeepseekV32IndexerMetadataBuilder (#38076) · 87f05d68
  Chauncey authored Mar 26, 2026
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  87f05d68
- [Misc] Optimized check to encapsulate both CUDA and ROCm platforms (#34549) · 36f6aede
  Andreas Karatzas authored Mar 25, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
```
  36f6aede
- Disable dual stream execution of input projection for Qwen3 (#38152) · 9704a5c3
  Xin Yang authored Mar 25, 2026
```
Signed-off-by: Xin Yang <xyangx@amazon.com>
```
  9704a5c3
- Fix minimax m2.5 nvfp4 kv scales weight loading (#37214) · 74056039
  Wei Zhao authored Mar 25, 2026
```
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
```
  74056039
- [Bugfix] Fix Qwen3.5-FP8 Weight Loading Error on TPU (#37348) · d7d51a7e
  Jacob Platin authored Mar 25, 2026
```
Signed-off-by: Jacob Platin <jacobplatin@google.com>
```
  d7d51a7e
- Various Transformers v5 fixes (#38127) · 3c3c0842
  Harry Mellor authored Mar 26, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  3c3c0842
25 Mar, 2026 28 commits
- [Cohere] Enable Cohere-Transcribe (#38120) · 7b54f60d
  Ekagra Ranjan authored Mar 25, 2026
```
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
```
  7b54f60d
- [ROCm]: Update rope+kvcache fusion conditions and disable custom op by default (#36716) · a0e8c740
  Rohan Potdar authored Mar 25, 2026
```
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
```
  a0e8c740
- [MultiModal] add support for numpy array embeddings (#38119) · 70a21528
  Guillaume Guy authored Mar 25, 2026
```
Signed-off-by: guillaume_guy <guillaume.guy@airbnb.com>
Signed-off-by: Guillaume Guy <guillaume.c.guy@gmail.com>
Co-authored-by: guillaume_guy <guillaume.guy@airbnb.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  70a21528
- [ROCm] Utilize persistent MLA kernel from AITER (#36574) · 978fc18b
  Sathish Sanjeevi authored Mar 25, 2026
```
Signed-off-by: Sathish Sanjeevi <sathish.krishnan.p.s@gmail.com>
```
  978fc18b
- [ROCm] Fix MoE kernel test failures on gfx950 (#37833) · 7d6917be
  Andreas Karatzas authored Mar 25, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
```
  7d6917be
- [Core][KV Connector] Remove use of num_cached_tokens in error handling (#38096) · e38817fa
  Mark McLoughlin authored Mar 25, 2026
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  e38817fa
- [Frontend] Move APIServerProcessManager target server fn (#38115) · 72cad44d
  Nick Hill authored Mar 25, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  72cad44d
- [Misc] Reorganize inputs (#35182) · ba2f0acc
  Cyrus Leung authored Mar 26, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ba2f0acc
- [MoE Kernel] Flashinfer nvfp4 cutedsl moe kernel integration (#38050) · 678b3c99
  Yongye Zhu authored Mar 25, 2026
  
  678b3c99
- [Refactor] Rename `WAITING_FOR_FSM` to `WAITING_FOR_STRUCTURED_OUTPUT_GRAMMAR` (#38048) · 1bf2ddd0
  Wentao Ye authored Mar 25, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  1bf2ddd0
- [Feature] EPLB Support for GPU Model Runner v2 (#37488) · d7e93e13
  Wentao Ye authored Mar 25, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
```
  d7e93e13
- [Feature] Support per-draft-model MoE backend via `--speculative-config` (#37880) · cd764301
  Andrii Skliar authored Mar 25, 2026
```
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: [Andrii Skliar] <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
```
  cd764301
- [KVTransfer][Mooncake] Add heterogeneous TP support for disaggregated P/D in... · b745e8b5
  yjz authored Mar 25, 2026
```
[KVTransfer][Mooncake] Add heterogeneous TP support for disaggregated P/D in MooncakeConnector (#36869)
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com>
```
  b745e8b5
- [Mypy] Better fixes for the `mypy` issues in `vllm/config` (#37902) · d215d1ef
  Harry Mellor authored Mar 25, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d215d1ef
- [CPU][UX][Perf] Enable tcmalloc by default (#37607) · 34d317dc
  Fadi Arafeh authored Mar 25, 2026
```
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
```
  34d317dc
- [Model] Add AutoWeightsLoader support for jais (#38074) · 7ac48fd3
  grYe99 authored Mar 25, 2026
```
Signed-off-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: grYe99 <guorongye99@gmail.com>
```
  7ac48fd3
- Fix Plamo 2/3 & LFM2 for Transformers v5 (#38090) · d6bb2a9d
  Harry Mellor authored Mar 25, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6bb2a9d
- Better weight tying check for multimodal models (#38035) · 1e673a43
  Harry Mellor authored Mar 25, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  1e673a43
- [Bugfix] Pass drafter quant_config to ParallelLMHead in Eagle3 (#37280) · a889b7f5
  Matthias Gehre authored Mar 25, 2026
```
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
```
  a889b7f5
- [ROCm][CI] Fix flaky Cohere/OpenAI embedding parity test (#37616) · f262a62a
  Andreas Karatzas authored Mar 25, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
```
  f262a62a
- [CI] Fix realtime WebSocket timeout deadlock and unhandled model validation errors (#37483) · 9ac2fcaf
  Andreas Karatzas authored Mar 25, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
```
  9ac2fcaf
- [Hardware][XPU] Align memory usage with cuda on xpu (#37029) · e9ae3f80
  Kunshang Ji authored Mar 25, 2026
```
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  e9ae3f80
- [XPU] support MLA model on Intel GPU (#37143) · 14771f71
  Kunshang Ji authored Mar 25, 2026
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  14771f71
- [ROCm] Attention selector reordering (#36702) · 189ddefb
  Gregory Shtrasberg authored Mar 25, 2026
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
```
  189ddefb
- [Revert] Remove CUDA torch fallbacks for... · 09c3dc91
  Chauncey authored Mar 25, 2026
```
[Revert] Remove CUDA torch fallbacks for fp8_mqa_logits/fp8_paged_mqa_logits_torch function (#37968)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  09c3dc91
- [Bugfix] Fix IndexError when accessing prev_tool_call_arr in OpenAIToolParser (#37958) · a32783bb
  Chauncey authored Mar 25, 2026
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  a32783bb
- [Performance] Auto-enable prefetch on NFS with RAM guard (#37673) · a93a53f8
  Artem Perevedentsev authored Mar 25, 2026
```
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
```
  a93a53f8
- [Bugfix][ROCm][MoE] Fix mxfp4 oracle regressions from #37128 (#37787) · 679c6a3e
  Andreas Karatzas authored Mar 24, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
```
  679c6a3e
24 Mar, 2026 1 commit
- [Perf] Disable inductor runtime asserts by default for serving perfor… (#37485) · 82580b10
  Terry Gao authored Mar 24, 2026
```
Signed-off-by: tianrengao <terrygao87@gmail.com>
Co-authored-by: Tianren Gao <tianren@fb.com>
```
  82580b10