Commits · 532a6cfccbaa1bc943512cca06b48d5d3500669d · OpenDAS / vllm_cscc

25 Sep, 2025 8 commits
- [mypy] Fix wrong type annotations related to tuple (#25660) · 2f171176
  Cyrus Leung authored Sep 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  2f171176
- [CI/Build] Fix flaky entrypoints test (#25663) · 0bcc3a16
  Cyrus Leung authored Sep 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0bcc3a16
- [V0 deprecation] Remove unreachable model_config.supported_tasks (#25642) · 7f570f1c
  wang.yuqi authored Sep 25, 2025
```
Signed-off-by: wang.yuqi <noooop@126.com>
```
  7f570f1c
- [Bugfix] Add triton.language.tensor placeholder (#25649) · 3c2b2cce
  Agata Dobrzyniewicz authored Sep 25, 2025
```
Signed-off-by: Agata Dobrzyniewicz <adobrzyniewicz@habana.ai>
```
  3c2b2cce
- Revert "[Performance] Move apply_w8a8_block_fp8_linear to an op class… (#25607) · 1260180c
  Tyler Michael Smith authored Sep 25, 2025
```
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
```
  1260180c
- Map CwmForCausalLM to llama and LlamaForCausalLM (#25611) · bc092ea8
  Jacob Kahn authored Sep 25, 2025
```
Signed-off-by: Jacob Kahn <jacobkahn1@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
```
  bc092ea8
- [Model] Add LongCat-Flash (#23991) · 845adb3e
  XuruiYang authored Sep 25, 2025
```
Signed-off-by: yangxurui <yangxurui@meituan.com>
Co-authored-by: yangxurui <yangxurui@meituan.com>
```
  845adb3e
- [Kernel] Support DCP for Triton backend (#25132) · 05c19485
  Wei Wei authored Sep 24, 2025
```
Signed-off-by: Wei Wei <wwei6@meta.com>
```
  05c19485
24 Sep, 2025 13 commits
- [Refactor] Use DeepGEMM Col Major TMA Aligned Tensor (#25517) · 1f291412
  Wentao Ye authored Sep 24, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  1f291412
- Support mnnvl all2allv from Flashinfer (#21003) · 54e42b72
  Shu Wang authored Sep 24, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
```
  54e42b72
- [Bugfix] Fix DeepSeekV31ToolParser to correctly parse multiple tools in... · e18b714b
  Tao Hui authored Sep 24, 2025
```
[Bugfix] Fix DeepSeekV31ToolParser to correctly parse multiple tools in non-streaming output (#25405)
Signed-off-by: taohui <taohui3@gmail.com>
```
  e18b714b
- [Bug] fix import and unit test (#25558) · 58c360d9
  Jonas M. Kübler authored Sep 24, 2025
```
Signed-off-by: Jonas M. Kübler <44084297+jmkuebler@users.noreply.github.com>
```
  58c360d9
- [V0 Deprecation] Remove max_seq_len_to_capture (#25543) · 2e19a848
  Woosuk Kwon authored Sep 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2e19a848
- [Misc]] Move processing context to multimodal directory (#25548) · 6488f348
  Cyrus Leung authored Sep 24, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6488f348
- [CI/Build] Fix v1 OOT registration test (#25547) · 27ec3c78
  Isotr0py authored Sep 24, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  27ec3c78
- [TPU][Bugfix] fix the missing apply_model in tpu worker (#25526) · 190c45a6
  Chengji Yao authored Sep 23, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  190c45a6
- [Bugfix] [Frontend] Cleanup gpt-oss non-streaming chat tool calls (#25514) · 5caaeb71
  Ben Browning authored Sep 23, 2025
```
Signed-off-by: Ben Browning <bbrownin@redhat.com>
```
  5caaeb71
- [Spec Decode] Enable FlashInfer Spec Decoding (#25196) · c30b405b
  Benjamin Chislett authored Sep 23, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: lhsjohn <huashuoli@tencent.com>
```
  c30b405b
- [Model] Support SeedOss Reason Parser (#24263) · be0bb568
  0xNullPath authored Sep 24, 2025
```
Signed-off-by: Yan Lu <luyan@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  be0bb568
- [BUG] Allows for RunAI Streamer and Torch.compile cache to be used together (#24922) · c8bde933
  ahao-anyscale authored Sep 23, 2025
```
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
```
  c8bde933
- Improve output when failing json.loads() on structured output test (#25483) · 7ad5e50a
  Doug Smith authored Sep 23, 2025
```
Signed-off-by: dougbtv <dosmith@redhat.com>
```
  7ad5e50a
23 Sep, 2025 19 commits

[BugFix] Fix OOM in vLLM replicas by ensuring consistent NCCL memory accounting (#25359) · abad204b
kourosh hakhamaneshi authored Sep 23, 2025
```
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
```
abad204b
[gpt-oss][bugfix] remove logic to require resp_ in ResponseAPI (#25428) · 95bc60e4
Andrew Xia authored Sep 23, 2025
```
Signed-off-by: Andrew Xia <axia@meta.com>
```
95bc60e4
[V0 Deprecation] Remove placeholder attn (#25510) · 969b4da3
Thomas Parnell authored Sep 24, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
969b4da3
[Core] Use KVCacheBlock as much as possible instead of dict[block_id, KVCacheBlock] (#24830) · 4f8c4b89
Jialin Ouyang authored Sep 23, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
4f8c4b89
[CI/Build] Fix and re-enable v1 PP test on CI (#25496) · ae002924
Isotr0py authored Sep 24, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
ae002924
Enable symmetric memory all reduce by default only enabling for TP (#25070) · 8bdd8b5c
Ilya Markov authored Sep 23, 2025
```
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
8bdd8b5c
[Speculators][Speculative Decoding] Fix gpt-oss eagle3 accuracy issue (#25406) · d5944d51
jiahanc authored Sep 23, 2025
```
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
```
d5944d51

[Performance] Move apply_w8a8_block_fp8_linear to an op class (#24666) · 63400259

ElizaWszola authored Sep 23, 2025


Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: ElizaWszola <elizaw.9289@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>

63400259

[core] add nccl symmetric memory for all reduce (#24532) · 8c1c81a3

Amir Samani authored Sep 23, 2025


Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

8c1c81a3

[ROCm] Add skinny gemm bias support for dtypes fp16,bf16,fp8 (#24988) · a3a78280

Hashem Hashemi authored Sep 23, 2025


Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
Signed-off-by: Hashem Hashemi <159079214+amd-hhashemi@users.noreply.github.com>

a3a78280

[V1][Kernel] Add triton implementation for `reshape_and_cache_flash` (#24503) · 100b630a

Burkhard Ringlein authored Sep 23, 2025

Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Chih-Chieh Yang <chih.chieh.yang@ibm.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

100b630a

Add backward compatibility for `GuidedDecodingParams` (#25422) · 875d6def
Harry Mellor authored Sep 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
875d6def

[Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) · cc1dc7ed

Lucas Wilkinson authored Sep 23, 2025


Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

cc1dc7ed

[V1] Remove V0 code paths for Hybrid models (#25400) · a903669e
Thomas Parnell authored Sep 23, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
a903669e

[Docs] NixlConnector quickstart guide (#24249) · da5e7e43

Peter Pan authored Sep 23, 2025


Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
Signed-off-by: Peter Pan <peter.pan@daocloud.io>
Signed-off-by: Nicolò Lucchesi<nicolo.lucchesi@gmail.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>

da5e7e43

[P/D] Support NIXL connector to disconnect during a clean shutdown (#24423) · f05a4f0e
Chauncey authored Sep 23, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
```
f05a4f0e
[CI/Build] Fix disabled v1 attention backend selection test (#25471) · b6a136b5
Isotr0py authored Sep 23, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
b6a136b5
[Core] Optimize LoRA weight loading (#25403) · 273690a5
Jee Jee Li authored Sep 23, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
273690a5
[Test]: Hermes tool parser stream output error in Qwen3 case (#25203) · 4322c553
Andreas Hartel authored Sep 23, 2025
```
Signed-off-by: Andreas Hartel <andreas.hartel@aleph-alpha.com>
```
4322c553