Commits · 63400259d05485330fb23635af892b3fce160dea · OpenDAS / vllm_cscc

23 Sep, 2025 18 commits

[Performance] Move apply_w8a8_block_fp8_linear to an op class (#24666) · 63400259

ElizaWszola authored Sep 23, 2025


Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: ElizaWszola <elizaw.9289@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>

63400259

[core] add nccl symmetric memory for all reduce (#24532) · 8c1c81a3

Amir Samani authored Sep 23, 2025


Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

8c1c81a3

[ROCm] Add skinny gemm bias support for dtypes fp16,bf16,fp8 (#24988) · a3a78280

Hashem Hashemi authored Sep 23, 2025


Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
Signed-off-by: Hashem Hashemi <159079214+amd-hhashemi@users.noreply.github.com>

a3a78280

[V1][Kernel] Add triton implementation for `reshape_and_cache_flash` (#24503) · 100b630a

Burkhard Ringlein authored Sep 23, 2025

Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Chih-Chieh Yang <chih.chieh.yang@ibm.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

100b630a

Add backward compatibility for `GuidedDecodingParams` (#25422) · 875d6def
Harry Mellor authored Sep 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
875d6def

[Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) · cc1dc7ed

Lucas Wilkinson authored Sep 23, 2025


Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

cc1dc7ed

[V1] Remove V0 code paths for Hybrid models (#25400) · a903669e
Thomas Parnell authored Sep 23, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
a903669e

[Docs] NixlConnector quickstart guide (#24249) · da5e7e43

Peter Pan authored Sep 23, 2025


Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
Signed-off-by: Peter Pan <peter.pan@daocloud.io>
Signed-off-by: Nicolò Lucchesi<nicolo.lucchesi@gmail.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>

da5e7e43

[P/D] Support NIXL connector to disconnect during a clean shutdown (#24423) · f05a4f0e
Chauncey authored Sep 23, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
```
f05a4f0e
[CI/Build] Fix disabled v1 attention backend selection test (#25471) · b6a136b5
Isotr0py authored Sep 23, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
b6a136b5
[Core] Optimize LoRA weight loading (#25403) · 273690a5
Jee Jee Li authored Sep 23, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
273690a5
[Test]: Hermes tool parser stream output error in Qwen3 case (#25203) · 4322c553
Andreas Hartel authored Sep 23, 2025
```
Signed-off-by: Andreas Hartel <andreas.hartel@aleph-alpha.com>
```
4322c553
[Misc] Move DP for ViT code inside model executor dir (#25459) · babad6e5
Cyrus Leung authored Sep 23, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
babad6e5
[Frontend] Add a new xml-based tool parser for qwen3-coder (#25028) · 9383cd6f
Zhikaiiii authored Sep 23, 2025
```
Signed-off-by: Zhikaiiii <1658973216@qq.com>
```
9383cd6f
[NIXL][OOT platform] support nixl_connector with oot platform and other nixl_backend (#25121) · 5774b0a1
Chendi.Xue authored Sep 22, 2025
```
Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>
```
5774b0a1
[V0 deprecation] Remove platform v1 controling interface (#25410) · 6fa78d8f
Isotr0py authored Sep 23, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
6fa78d8f
[Bugfix] Fix missing `clear_connector_metadata` (#25397) · 09019703
Nicolò Lucchesi authored Sep 23, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
09019703
[Bugfix] fix custom op test (#25429) · d588cd24
Luka Govedič authored Sep 22, 2025
```
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
```
d588cd24

22 Sep, 2025 12 commits

[Frontend] Responses API MCP tools for built in tools and to pass through headers (#24628) · 45d7d852

Alec S authored Sep 22, 2025


Signed-off-by: Alec Solder <alecs@fb.com>
Signed-off-by: Alec S <10566873+alecsolder@users.noreply.github.com>
Co-authored-by: Alec Solder <alecs@fb.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>

45d7d852

[CI/Build] Skip Qwen3-VL initialization tests until models are actually released (#25394) · f552d5e5
Cyrus Leung authored Sep 23, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
f552d5e5
[KV offload][5/N] Add `CPUOffloadingSpec` (#24251) · 8db29392
Or Ozeri authored Sep 22, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
8db29392

[torch.compile] Cleanup compilation tests and custom passes, add debug utils,... · d5e0fca2

Luka Govedič authored Sep 22, 2025


[torch.compile] Cleanup compilation tests and custom passes, add debug utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604) (#24542)
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

d5e0fca2

[DP] support torchrun external launcher with Data Parallelism (#24899) · 922979bf

Lucia Fang authored Sep 22, 2025


Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

922979bf

[CLI env var] Add VLLM_FLASH_ATTN_MAX_NUM_SPLITS_FOR_CUDA_GRAPH in env variables (#25274) · cfbee3d0

Daisy-Ma-coder authored Sep 22, 2025


Signed-off-by: qqma <qqma@amazon.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: qqma <qqma@amazon.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

cfbee3d0

[TPU] Deprecate `xm.mark_step` in favor of ``torch_xla.sync` (#25254) · 4cf71cc8
Nicolò Lucchesi authored Sep 22, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
```
4cf71cc8
[V0 Deprecation] Remove `MultiModalPlaceholderMap` (#25366) · f92d9526
Cyrus Leung authored Sep 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
f92d9526
[V0 Deprecation] Remove V0-only methods in multi-modal registry (#25362) · 6d0b827c
Cyrus Leung authored Sep 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
6d0b827c

[Bugfix] Fix hermes tool parser handling of non-string argument types (#22002) · 0eecb316

WeiQing Chen authored Sep 22, 2025


Signed-off-by: wangzi <3220100013@zju.edu.cn>
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: wangzi <3220100013@zju.edu.cn>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>

0eecb316

[Docs] GSM8K Accuracy Evaluation doc update (#25360) · 793be8d0
WeiQing Chen authored Sep 22, 2025
```
Signed-off-by: David Chen <530634352@qq.com>
```
793be8d0

[Model] Support Dots OCR (#24645) · 7b57a433

Roger Wang authored Sep 21, 2025


Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: yinz-aizip <yinz@aizip.ai>

7b57a433

21 Sep, 2025 10 commits
- Multimodal - audio tests (#25285) · 5aeb9254
  Deboleina authored Sep 21, 2025
```
Signed-off-by: Debolina Roy <debroy@redhat.com>
```
  5aeb9254
- [Bugfix][V0 Deprecation][CI] use async mock and await for async method (#25325) · 04d37523
  Yang Liu authored Sep 21, 2025
```
Signed-off-by: Yang <lymailforjob@gmail.com>
```
  04d37523
- Remove V0 attention backends (#25351) · bc6e542d
  Woosuk Kwon authored Sep 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  bc6e542d
- [V0 Deprecation] Remove V0 sampling metadata (#25345) · 1c3ffdbe
  Woosuk Kwon authored Sep 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
  1c3ffdbe
- feat: Enable engine-level arguments with speculators models (#25250) · c438b295
  Rahul Tuli authored Sep 21, 2025
```
Signed-off-by: Rahul Tuli <rtuli@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
```
  c438b295
- [V0 Deprecation] Remove async_output_proc, preemption mode, delay factor (#25334) · 0ff8ebb2
  Woosuk Kwon authored Sep 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  0ff8ebb2
- [V0 Deprecation] Remove V0 Sequence class & Sampler (#25332) · 26e673fe
  Woosuk Kwon authored Sep 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
  26e673fe
- [V1] Add sliding window support to Flex Attention backend (#24089) · cf56cf78
  Isotr0py authored Sep 21, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  cf56cf78
- [CI] Skip tests failing on main (#25326) · 72dd1595
  Woosuk Kwon authored Sep 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  72dd1595
- [Chore] Remove unused sampler in models (#25324) · 572ddf83
  Woosuk Kwon authored Sep 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  572ddf83