Commits · d747c2ef18f9d9174bf40c58ae313ab31cdba305 · OpenDAS / vllm_cscc

24 Sep, 2025 4 commits
- [Spec Decode] Enable FlashInfer Spec Decoding (#25196) · c30b405b
  Benjamin Chislett authored Sep 23, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: lhsjohn <huashuoli@tencent.com>
```
  c30b405b
- [Model] Support SeedOss Reason Parser (#24263) · be0bb568
  0xNullPath authored Sep 24, 2025
```
Signed-off-by: Yan Lu <luyan@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  be0bb568
- [BUG] Allows for RunAI Streamer and Torch.compile cache to be used together (#24922) · c8bde933
  ahao-anyscale authored Sep 23, 2025
```
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
```
  c8bde933
- Improve output when failing json.loads() on structured output test (#25483) · 7ad5e50a
  Doug Smith authored Sep 23, 2025
```
Signed-off-by: dougbtv <dosmith@redhat.com>
```
  7ad5e50a
23 Sep, 2025 25 commits
- [BugFix] Fix OOM in vLLM replicas by ensuring consistent NCCL memory accounting (#25359) · abad204b
  kourosh hakhamaneshi authored Sep 23, 2025
```
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
```
  abad204b
- [gpt-oss][bugfix] remove logic to require resp_ in ResponseAPI (#25428) · 95bc60e4
  Andrew Xia authored Sep 23, 2025
```
Signed-off-by: Andrew Xia <axia@meta.com>
```
  95bc60e4
- [V0 Deprecation] Remove placeholder attn (#25510) · 969b4da3
  Thomas Parnell authored Sep 24, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  969b4da3
- [Core] Use KVCacheBlock as much as possible instead of dict[block_id, KVCacheBlock] (#24830) · 4f8c4b89
  Jialin Ouyang authored Sep 23, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  4f8c4b89
- [CI/Build] Fix and re-enable v1 PP test on CI (#25496) · ae002924
  Isotr0py authored Sep 24, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  ae002924
- Enable symmetric memory all reduce by default only enabling for TP (#25070) · 8bdd8b5c
  Ilya Markov authored Sep 23, 2025
```
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  8bdd8b5c
- [Speculators][Speculative Decoding] Fix gpt-oss eagle3 accuracy issue (#25406) · d5944d51
  jiahanc authored Sep 23, 2025
```
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
```
  d5944d51
- [Performance] Move apply_w8a8_block_fp8_linear to an op class (#24666) · 63400259
  ElizaWszola authored Sep 23, 2025
```
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: ElizaWszola <elizaw.9289@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
```
  63400259
- [core] add nccl symmetric memory for all reduce (#24532) · 8c1c81a3
  Amir Samani authored Sep 23, 2025
```
Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  8c1c81a3
- [ROCm] Add skinny gemm bias support for dtypes fp16,bf16,fp8 (#24988) · a3a78280
  Hashem Hashemi authored Sep 23, 2025
```
Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
Signed-off-by: Hashem Hashemi <159079214+amd-hhashemi@users.noreply.github.com>
```
  a3a78280
- [V1][Kernel] Add triton implementation for `reshape_and_cache_flash` (#24503) · 100b630a
  Burkhard Ringlein authored Sep 23, 2025
```
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Chih-Chieh Yang <chih.chieh.yang@ibm.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
```
  100b630a
- Add backward compatibility for `GuidedDecodingParams` (#25422) · 875d6def
  Harry Mellor authored Sep 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  875d6def
- [Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) · cc1dc7ed
  Lucas Wilkinson authored Sep 23, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  cc1dc7ed
- [V1] Remove V0 code paths for Hybrid models (#25400) · a903669e
  Thomas Parnell authored Sep 23, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  a903669e
- [Docs] NixlConnector quickstart guide (#24249) · da5e7e43
  Peter Pan authored Sep 23, 2025
```
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
Signed-off-by: Peter Pan <peter.pan@daocloud.io>
Signed-off-by: Nicolò Lucchesi<nicolo.lucchesi@gmail.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
```
  da5e7e43
- [P/D] Support NIXL connector to disconnect during a clean shutdown (#24423) · f05a4f0e
  Chauncey authored Sep 23, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
```
  f05a4f0e
- [CI/Build] Fix disabled v1 attention backend selection test (#25471) · b6a136b5
  Isotr0py authored Sep 23, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  b6a136b5
- [Core] Optimize LoRA weight loading (#25403) · 273690a5
  Jee Jee Li authored Sep 23, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  273690a5
- [Test]: Hermes tool parser stream output error in Qwen3 case (#25203) · 4322c553
  Andreas Hartel authored Sep 23, 2025
```
Signed-off-by: Andreas Hartel <andreas.hartel@aleph-alpha.com>
```
  4322c553
- [Misc] Move DP for ViT code inside model executor dir (#25459) · babad6e5
  Cyrus Leung authored Sep 23, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  babad6e5
- [Frontend] Add a new xml-based tool parser for qwen3-coder (#25028) · 9383cd6f
  Zhikaiiii authored Sep 23, 2025
```
Signed-off-by: Zhikaiiii <1658973216@qq.com>
```
  9383cd6f
- [NIXL][OOT platform] support nixl_connector with oot platform and other nixl_backend (#25121) · 5774b0a1
  Chendi.Xue authored Sep 22, 2025
```
Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>
```
  5774b0a1
- [V0 deprecation] Remove platform v1 controling interface (#25410) · 6fa78d8f
  Isotr0py authored Sep 23, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  6fa78d8f
- [Bugfix] Fix missing `clear_connector_metadata` (#25397) · 09019703
  Nicolò Lucchesi authored Sep 23, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  09019703
- [Bugfix] fix custom op test (#25429) · d588cd24
  Luka Govedič authored Sep 22, 2025
```
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
```
  d588cd24
22 Sep, 2025 11 commits

[Frontend] Responses API MCP tools for built in tools and to pass through headers (#24628) · 45d7d852

Alec S authored Sep 22, 2025


Signed-off-by: Alec Solder <alecs@fb.com>
Signed-off-by: Alec S <10566873+alecsolder@users.noreply.github.com>
Co-authored-by: Alec Solder <alecs@fb.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>

45d7d852

[CI/Build] Skip Qwen3-VL initialization tests until models are actually released (#25394) · f552d5e5
Cyrus Leung authored Sep 23, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
f552d5e5
[KV offload][5/N] Add `CPUOffloadingSpec` (#24251) · 8db29392
Or Ozeri authored Sep 22, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
8db29392

[torch.compile] Cleanup compilation tests and custom passes, add debug utils,... · d5e0fca2

Luka Govedič authored Sep 22, 2025


[torch.compile] Cleanup compilation tests and custom passes, add debug utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604) (#24542)
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

d5e0fca2

[DP] support torchrun external launcher with Data Parallelism (#24899) · 922979bf

Lucia Fang authored Sep 22, 2025


Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

922979bf

[CLI env var] Add VLLM_FLASH_ATTN_MAX_NUM_SPLITS_FOR_CUDA_GRAPH in env variables (#25274) · cfbee3d0

Daisy-Ma-coder authored Sep 22, 2025


Signed-off-by: qqma <qqma@amazon.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: qqma <qqma@amazon.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

cfbee3d0

[TPU] Deprecate `xm.mark_step` in favor of ``torch_xla.sync` (#25254) · 4cf71cc8
Nicolò Lucchesi authored Sep 22, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
```
4cf71cc8
[V0 Deprecation] Remove `MultiModalPlaceholderMap` (#25366) · f92d9526
Cyrus Leung authored Sep 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
f92d9526
[V0 Deprecation] Remove V0-only methods in multi-modal registry (#25362) · 6d0b827c
Cyrus Leung authored Sep 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
6d0b827c

[Bugfix] Fix hermes tool parser handling of non-string argument types (#22002) · 0eecb316

WeiQing Chen authored Sep 22, 2025


Signed-off-by: wangzi <3220100013@zju.edu.cn>
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: wangzi <3220100013@zju.edu.cn>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>

0eecb316

[Docs] GSM8K Accuracy Evaluation doc update (#25360) · 793be8d0
WeiQing Chen authored Sep 22, 2025
```
Signed-off-by: David Chen <530634352@qq.com>
```
793be8d0