Commits · 7c0cf3bcd0867b5420e6ad4f2ff6b2b25e73b022 · OpenDAS / vllm_cscc · GitLab

19 Mar, 2026 29 commits
- Cap the number of API servers to 1 when using Elastic EP. (#37466) · 7c0cf3bc
  Sage Moore authored Mar 19, 2026
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  7c0cf3bc
- Stop bench CLI from recursively casting all configs to `dict` (#37559) · 572b4329
  Harry Mellor authored Mar 19, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  572b4329
- [Misc] Clean up processing logic (#37541) · 9515c208
  Cyrus Leung authored Mar 19, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9515c208
- [Bugfix] Add Kimi-K2.5 reasoning/tool parser aliases and tool_call_id support (#37438) · c63ca2b2
  DorBernsohn authored Mar 19, 2026
```
Signed-off-by: DorBernsohn <dor.bernsohn@gmail.com>
```
  c63ca2b2
- [CI] Merge `cleanup_pr_body.yml` and `reminder_comment.yml` (#37552) · a32eaf5b
  Harry Mellor authored Mar 19, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a32eaf5b
- Fix KV Offloading + MLA AssertionError by using num_kv_heads=1 in cpu… (#37536) · e390742c
  XueLiang Yang authored Mar 19, 2026
```
Signed-off-by: xueliangyang-oeuler <yxl546827391@gmail.com>
Co-authored-by: xueliangyang-oeuler <yxl546827391@gmail.com>
```
  e390742c
- [Model] Remove unnecessary `get_language_model` (#37545) · 7a6ebcbf
  Cyrus Leung authored Mar 19, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  7a6ebcbf
- [CI/Build] Split out MM pooling tests (#37542) · c7bc12c2
  Cyrus Leung authored Mar 19, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c7bc12c2
- [Docs] Reorganize pooling docs. (#35592) · f9e2a383
  wang.yuqi authored Mar 19, 2026
```
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  f9e2a383
- Don't log `exc_info` when vLLM tries to doenload a file that doesn't exist (#37458) · 4426447b
  Harry Mellor authored Mar 19, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  4426447b
- [Bugfix] Avoid more OpenMP thread reallocation in CPU torch compile (#37538) · 3322e264
  Li, Jiang authored Mar 19, 2026
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  3322e264
- [Bugfix] Fix Nemotron Parse loading (#37407) · 765e4610
  Cyrus Leung authored Mar 19, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  765e4610
- [Bugfix][ROCm] Fix MoRI + AITER FP8 dispatch compatibility for defer_input_quant (#37418) · 6a9cceb2
  Duyi-Wang authored Mar 19, 2026
```
Signed-off-by: Duyi-Wang <duyi.wang@amd.com>
```
  6a9cceb2
- fix(cpu): add null check for aligned_alloc in ScratchPadManager (#37369) · 199f9141
  yassha authored Mar 19, 2026
```
Signed-off-by: yassha <50112520+yassha@users.noreply.github.com>
```
  199f9141
- [MISC] fix pin_memory=torch.cuda.is_available(), use is_pin_memory_available (#37415) · ca21483b
  Kunshang Ji authored Mar 19, 2026
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  ca21483b
- [CI] Fix wrong path test file, missing `rlhf_async_new_apis.py` (#37532) · da70c87e
  TJian authored Mar 19, 2026
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  da70c87e
- Support temporal compression for Nemotron-3-VL videos (#36808) · 0b6d5262
  Collin McCarthy authored Mar 19, 2026
```
Signed-off-by: Collin McCarthy <cmccarthy@nvidia.com>
```
  0b6d5262
- [Perf] Fix slow hasattr in CUDAGraphWrapper.__getattr__ (#37425) · d3cc3795
  Ziming Huang authored Mar 19, 2026
```
Signed-off-by: 智鸣 <hzm414167@alibaba-inc.com>
```
  d3cc3795
- fix(anthropic): remove non-standard 'data: [DONE]' from Anthropic streaming (#37510) · 354cd580
  cdpath authored Mar 19, 2026
```
Signed-off-by: cdpath <cdpath@outlook.com>
```
  354cd580
- [SSM/Mamba] Follow-up: N-1 prefill for P/D disaggregation (#37310) · d49f2731
  zhanqiuhu authored Mar 19, 2026
  
  d49f2731
- [Refactor] Relocate endpoint tests to mirror serving code directory structure (#37504) · b21d3843
  Flora Feng authored Mar 19, 2026
```
Signed-off-by: sfeng33 <4florafeng@gmail.com>
```
  b21d3843
- [ROCm] issue management - request information for bug issues on ROCm (#37009) · e3126cd1
  Hongxia Yang authored Mar 18, 2026
```
Signed-off-by: Hongxia Yang <hongxiay.yang@amd.com>
```
  e3126cd1
- [Perf] Optimize token_embed for pooling models, 1.0% token throughput improvement (#37347) · e37ff5b5
  Wentao Ye authored Mar 18, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  e37ff5b5
- [bug] Fix deadlock with pause resume and collective_rpc (#37024) · 6accb21f
  Aaron Hao authored Mar 18, 2026
```
Signed-off-by: hao-aaron <ahao@anyscale.com>
```
  6accb21f
- [Model Runner V2] Spec decode rejection sampler logprobs support (#37237) · 053f3b63
  Giancarlo Delfin authored Mar 18, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  053f3b63
- [BUG] Exclude SKIP_TENSORS from get_layer_size() + new weight sync example for dpep (#37334) · 5f82706a
  Aaron Hao authored Mar 18, 2026
```
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
```
  5f82706a
- [EPLB] Simplify EPLB rearrange by only returning one map (#36267) · c32a58cc
  Sage Moore authored Mar 18, 2026
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  c32a58cc
- [Bugfix] Zero-init MLA attention output buffers to prevent NaN from CUDA graph padding (#37442) · ef2c4f77
  Elvir Crnčević authored Mar 19, 2026
```
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
```
  ef2c4f77
- [XPU]Unify xpu test dependencies in dockerfile.xpu (#36477) · 9dade5da
  sihao_li authored Mar 19, 2026
```
Signed-off-by: sihao.li <sihao.li@intel.com>
```
  9dade5da
18 Mar, 2026 11 commits
- [Bugfix] Expand quantization method support in perf metrics (#37231) · 828f862a
  Thillai Chithambaram authored Mar 18, 2026
```
Signed-off-by: Thillai Chithambaram <thillaichithambaram.a@gmail.com>
```
  828f862a
- [Bugfix] Fix KV scales inconsistency in fp8 MLA & FlashInfer kv_cache_dtype... · 577df69b
  Andy Lo authored Mar 18, 2026
```
[Bugfix] Fix KV scales inconsistency in fp8 MLA & FlashInfer kv_cache_dtype "auto" leading to gibberish (#37054)
Signed-off-by: Andy Lo <andy@mistral.ai>
```
  577df69b
- [Model Runner V2] Spec decode rejection sampler greedy support (#37238) · 04244fd0
  Giancarlo Delfin authored Mar 18, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  04244fd0
- [Bugfix] Remove assertion for NVFP4 scale dynamic range (#37465) · 9482b0b0
  Michael Goin authored Mar 18, 2026
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
```
  9482b0b0
- [LoRA][BugFix] Fix skipped LoRA adapters for Mistral3 (#36928) · 5bc1da14
  Woosuk Kwon authored Mar 18, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  5bc1da14
- fix(worker): optimize swap_states to copy only active token prefixes (#34733) · 00910171
  Philip Ottesen authored Mar 18, 2026
```
Signed-off-by: Philip Ottesen <phiott256@gmail.com>
```
  00910171
- [V0 Deprecation] Deprecate virtual engine (#37195) · 0d81a1fe
  Wentao Ye authored Mar 18, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  0d81a1fe
- chunk parakeet into 30s clips to prevent OOMs on long audios (#36671) · 6ae4c8d6
  Netanel Haber authored Mar 18, 2026
```
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
```
  6ae4c8d6
- [Bugfix] Fix ROCm crash in qwen3_next multi-stream events (#36795) (#37427) · a913b612
  JartX authored Mar 18, 2026
```
Signed-off-by: JartX <sagformas@epdcenter.es>
```
  a913b612
- Fix models which use `layer_type_validation` for Transformers v5 (#37398) · 5ce2d10e
  Harry Mellor authored Mar 18, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  5ce2d10e
- [Bugfix] Fix incorrect use of merge_size in Qwen3-VL video timestamp calculation (#37439) · 738d0a28
  Chengyu Fang authored Mar 19, 2026
```
Signed-off-by: chengyufang <cnyvfang@outlook.com>
```
  738d0a28