Commits · 96266f119bb93516703328f9e37ec99cce45f792 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "bea23536f627b2b4153f2e672753b6034b78dedb"

19 Mar, 2026 22 commits
- [LoRA] Minor improvements to LoRA log (#37557) · 96266f11
  Jee Jee Li authored Mar 19, 2026
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  96266f11
- Cap the number of API servers to 1 when using Elastic EP. (#37466) · 7c0cf3bc
  Sage Moore authored Mar 19, 2026
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  7c0cf3bc
- Stop bench CLI from recursively casting all configs to `dict` (#37559) · 572b4329
  Harry Mellor authored Mar 19, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  572b4329
- [Misc] Clean up processing logic (#37541) · 9515c208
  Cyrus Leung authored Mar 19, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9515c208
- [Bugfix] Add Kimi-K2.5 reasoning/tool parser aliases and tool_call_id support (#37438) · c63ca2b2
  DorBernsohn authored Mar 19, 2026
```
Signed-off-by: DorBernsohn <dor.bernsohn@gmail.com>
```
  c63ca2b2
- Fix KV Offloading + MLA AssertionError by using num_kv_heads=1 in cpu… (#37536) · e390742c
  XueLiang Yang authored Mar 19, 2026
```
Signed-off-by: xueliangyang-oeuler <yxl546827391@gmail.com>
Co-authored-by: xueliangyang-oeuler <yxl546827391@gmail.com>
```
  e390742c
- [Model] Remove unnecessary `get_language_model` (#37545) · 7a6ebcbf
  Cyrus Leung authored Mar 19, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  7a6ebcbf
- Don't log `exc_info` when vLLM tries to doenload a file that doesn't exist (#37458) · 4426447b
  Harry Mellor authored Mar 19, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  4426447b
- [Bugfix] Avoid more OpenMP thread reallocation in CPU torch compile (#37538) · 3322e264
  Li, Jiang authored Mar 19, 2026
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  3322e264
- [Bugfix] Fix Nemotron Parse loading (#37407) · 765e4610
  Cyrus Leung authored Mar 19, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  765e4610
- [Bugfix][ROCm] Fix MoRI + AITER FP8 dispatch compatibility for defer_input_quant (#37418) · 6a9cceb2
  Duyi-Wang authored Mar 19, 2026
```
Signed-off-by: Duyi-Wang <duyi.wang@amd.com>
```
  6a9cceb2
- [MISC] fix pin_memory=torch.cuda.is_available(), use is_pin_memory_available (#37415) · ca21483b
  Kunshang Ji authored Mar 19, 2026
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  ca21483b
- Support temporal compression for Nemotron-3-VL videos (#36808) · 0b6d5262
  Collin McCarthy authored Mar 19, 2026
```
Signed-off-by: Collin McCarthy <cmccarthy@nvidia.com>
```
  0b6d5262
- [Perf] Fix slow hasattr in CUDAGraphWrapper.__getattr__ (#37425) · d3cc3795
  Ziming Huang authored Mar 19, 2026
```
Signed-off-by: 智鸣 <hzm414167@alibaba-inc.com>
```
  d3cc3795
- fix(anthropic): remove non-standard 'data: [DONE]' from Anthropic streaming (#37510) · 354cd580
  cdpath authored Mar 19, 2026
```
Signed-off-by: cdpath <cdpath@outlook.com>
```
  354cd580
- [SSM/Mamba] Follow-up: N-1 prefill for P/D disaggregation (#37310) · d49f2731
  zhanqiuhu authored Mar 19, 2026
  
  d49f2731
- [Perf] Optimize token_embed for pooling models, 1.0% token throughput improvement (#37347) · e37ff5b5
  Wentao Ye authored Mar 18, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  e37ff5b5
- [bug] Fix deadlock with pause resume and collective_rpc (#37024) · 6accb21f
  Aaron Hao authored Mar 18, 2026
```
Signed-off-by: hao-aaron <ahao@anyscale.com>
```
  6accb21f
- [Model Runner V2] Spec decode rejection sampler logprobs support (#37237) · 053f3b63
  Giancarlo Delfin authored Mar 18, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  053f3b63
- [BUG] Exclude SKIP_TENSORS from get_layer_size() + new weight sync example for dpep (#37334) · 5f82706a
  Aaron Hao authored Mar 18, 2026
```
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
```
  5f82706a
- [EPLB] Simplify EPLB rearrange by only returning one map (#36267) · c32a58cc
  Sage Moore authored Mar 18, 2026
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  c32a58cc
- [Bugfix] Zero-init MLA attention output buffers to prevent NaN from CUDA graph padding (#37442) · ef2c4f77
  Elvir Crnčević authored Mar 19, 2026
```
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
```
  ef2c4f77
18 Mar, 2026 18 commits
- [Bugfix] Expand quantization method support in perf metrics (#37231) · 828f862a
  Thillai Chithambaram authored Mar 18, 2026
```
Signed-off-by: Thillai Chithambaram <thillaichithambaram.a@gmail.com>
```
  828f862a
- [Bugfix] Fix KV scales inconsistency in fp8 MLA & FlashInfer kv_cache_dtype... · 577df69b
  Andy Lo authored Mar 18, 2026
```
[Bugfix] Fix KV scales inconsistency in fp8 MLA & FlashInfer kv_cache_dtype "auto" leading to gibberish (#37054)
Signed-off-by: Andy Lo <andy@mistral.ai>
```
  577df69b
- [Model Runner V2] Spec decode rejection sampler greedy support (#37238) · 04244fd0
  Giancarlo Delfin authored Mar 18, 2026
```
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
```
  04244fd0
- [Bugfix] Remove assertion for NVFP4 scale dynamic range (#37465) · 9482b0b0
  Michael Goin authored Mar 18, 2026
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
```
  9482b0b0
- [LoRA][BugFix] Fix skipped LoRA adapters for Mistral3 (#36928) · 5bc1da14
  Woosuk Kwon authored Mar 18, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
```
  5bc1da14
- fix(worker): optimize swap_states to copy only active token prefixes (#34733) · 00910171
  Philip Ottesen authored Mar 18, 2026
```
Signed-off-by: Philip Ottesen <phiott256@gmail.com>
```
  00910171
- [V0 Deprecation] Deprecate virtual engine (#37195) · 0d81a1fe
  Wentao Ye authored Mar 18, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  0d81a1fe
- chunk parakeet into 30s clips to prevent OOMs on long audios (#36671) · 6ae4c8d6
  Netanel Haber authored Mar 18, 2026
```
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
```
  6ae4c8d6
- [Bugfix] Fix ROCm crash in qwen3_next multi-stream events (#36795) (#37427) · a913b612
  JartX authored Mar 18, 2026
```
Signed-off-by: JartX <sagformas@epdcenter.es>
```
  a913b612
- Fix models which use `layer_type_validation` for Transformers v5 (#37398) · 5ce2d10e
  Harry Mellor authored Mar 18, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  5ce2d10e
- [Bugfix] Fix incorrect use of merge_size in Qwen3-VL video timestamp calculation (#37439) · 738d0a28
  Chengyu Fang authored Mar 19, 2026
```
Signed-off-by: chengyufang <cnyvfang@outlook.com>
```
  738d0a28
- [bugfix][async scheduling] fix extra cuda context in device 0 with EP/DP (#37449) · 70b81c4f
  youkaichao authored Mar 19, 2026
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  70b81c4f
- [Model] Remove unnecessary processor definition for Nemotron Parse (#37456) · 7476d148
  Cyrus Leung authored Mar 19, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  7476d148
- [Misc] Clean up model registry (#37457) · f3732bd9
  Cyrus Leung authored Mar 19, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f3732bd9
- [Perf] Add tuned triton moe config for Qwen3.5 H200, 9.9% E2E throughput improvement (#37340) · 0ef7f790
  Wentao Ye authored Mar 18, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  0ef7f790
- [kv_offload+HMA][2/N]: Support multiple KV groups in GPULoadStoreSpec (#36642) · 5dd8df07
  Or Ozeri authored Mar 18, 2026
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  5dd8df07
- Add API docs link if the CLI arg is a config class (#37432) · 39bfb57b
  Harry Mellor authored Mar 18, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  39bfb57b
- Adding deterministic lora benchmarking to vLLM Bench (#36057) · c9d838fc
  RonaldBXu authored Mar 18, 2026
```
Signed-off-by: Ubuntu <ubuntu@ip-172-31-43-201.ap-northeast-1.compute.internal>
Signed-off-by: Ronald Xu <ronaldxu@amazon.com>
```
  c9d838fc