Commits · 2f99f2f506f919922ee699350204a176f8f87c13 · OpenDAS / vllm_cscc

08 Oct, 2025 13 commits
- Tidy `vllm/config/__init__.py` to only add classes and functions (#26405) · 2f99f2f5
  Harry Mellor authored Oct 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  2f99f2f5
- [Model] Allow passing custom number of max tiles to Nano 2 VL (#26403) · f9582fd8
  Eugene Khvedchenya authored Oct 08, 2025
```
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
```
  f9582fd8
- [Misc] add `usedforsecurity=False` in md5 hash call (#26357) · f377333b
  Daniele authored Oct 08, 2025
```
Signed-off-by: Daniele Trifirò <dtrifiro@redhat.com>
```
  f377333b
- [Feature] Enable E8M0 by Default on Hopper for DeepGEMM, 5% E2E throughput improvement (#26197) · f8607863
  Wentao Ye authored Oct 08, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  f8607863
- [TPU] Rename tpu_commons to tpu_inference (#26279) · 335b28f7
  Utkarsh Sharma authored Oct 08, 2025
```
Signed-off-by: Utkarsh Sharma <utksharma@google.com>
Co-authored-by: Utkarsh Sharma <utksharma@google.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
```
  335b28f7
- fix[DP][v1]: Prevent hangs from mismatched worker configurations (#26218) · 5e65d6b2
  Ayush Satyam authored Oct 08, 2025
```
Signed-off-by: Ayush Satyam <ayushsatyam146@gmail.com>
```
  5e65d6b2
- [Bugfix] Incorrect MM data format in `vllm bench throughput` (#26395) · 0d4f48fa
  Cyrus Leung authored Oct 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0d4f48fa
- Add gather_indexer_k_quant_cache kernel (#25931) · 127c8b78
  Barry Kang authored Oct 08, 2025
```
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Signed-off-by: Simon Mo <simon.mo@hey.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
```
  127c8b78
- fix(v1/kv_cache): resolve async KV transfer bug in cascade attention (#23485) · cd989054
  Ayush Satyam authored Oct 08, 2025
```
Signed-off-by: Ayush Satyam <ayushsatyam146@gmail.com>
```
  cd989054
- [Core] Simplify setting new_token_ids in CachedRequestData (#26388) · 067da2d1
  Nick Hill authored Oct 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  067da2d1
- Add SwigluOAI implementation for CPUFusedMOE (#26347) · 046118b9
  isharif168 authored Oct 08, 2025
```
Signed-off-by: Sharif Inamdar <sharif.inamdar@arm.com>
```
  046118b9
- [torchao] safetensors integration (#25969) · b32260ab
  liangel-02 authored Oct 07, 2025
```
Signed-off-by: Angel Li <liangel@meta.com>
```
  b32260ab
- [Misc] Clean up cruft from previous FlashMLA sparse implementation (#26125) · f80e7866
  Lucas Wilkinson authored Oct 07, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  f80e7866
07 Oct, 2025 23 commits

Revert #24446 and #26168 (#26332) · 31a4b3e6
Thomas Parnell authored Oct 08, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
31a4b3e6

[Bugfix] Fix MTP+FlashInfer crash when trtllm kernels are available but disabled (#26361) · caf8b1c0

Benjamin Chislett authored Oct 07, 2025

Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

caf8b1c0

[Spec Decode] Enable efficient speculative decoding with FlashInfer-MLA (#25984) · 3d1f6761
Benjamin Chislett authored Oct 07, 2025
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
3d1f6761

[V1] Logit processors for rejection sampler (#19482) · 6ebaf43e

Sergei Skvortsov authored Oct 07, 2025


Signed-off-by: southfreebird <yvorott@gmail.com>
Signed-off-by: Sergei Skvortsov <sergeyskv@nebius.com>
Signed-off-by: Sergei Skvortsov <yvorott@gmail.com>
Co-authored-by: Sergei Skvortsov <sergeyskv@nebius.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

6ebaf43e

[Frontend] CompilationConfig overhaul (#20283): deprecate use_inductor in... · 0c824fc4

Morrison Turnansky authored Oct 07, 2025


[Frontend] CompilationConfig overhaul (#20283): deprecate use_inductor in favor of backend, simplify custom_ops (#26113)
Signed-off-by: morrison-turnansky <mturnans@redhat.com>
Signed-off-by: Morrison Turnansky <mturnans@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>

0c824fc4

[Bugfix] Add missing sink tensor into flash attn cascade attn implementation (#26325) · eb577e46
Pei-Lun Liao authored Oct 07, 2025

eb577e46
[Bug] Fix Shape Validation for Fallback while Enabling E8M0 for DeepGEMM (#26322) · 8f36850f
Wentao Ye authored Oct 07, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
8f36850f
[deepseek] add EP8 FusedMOE config for H200 and B200 (#26331) · 29fd2662
Chen Zhang authored Oct 08, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
29fd2662
[Model] Lfm2Moe (#26344) · 320feae6
Paul Pak authored Oct 08, 2025
```
Signed-off-by: Paul Pak <paulpak58@gmail.com>
```
320feae6
[V0 Deprecation] Remove `VLLM_USE_V1` from tests (#26341) · 1e4ecca1
Cyrus Leung authored Oct 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
1e4ecca1
[Misc] Move `LRUCache` into its own file (#26342) · c0a7b89d
Cyrus Leung authored Oct 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
c0a7b89d

[Model] Add support for ModernBertForTokenClassification (#26340) · 6f59beaf

antrec authored Oct 07, 2025

Signed-off-by: Antoine Recanati Le Goat <antoine.recanati@sancare.fr>
Signed-off-by: antrec <antoine.recanati@gmail.com>
Co-authored-by: Antoine Recanati Le Goat <antoine.recanati@sancare.fr>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

6f59beaf

[Feature][OCP MX] Support mxfp6 and mixed mxfp6-mxfp4 (#21166) · 41f1cf38
fxmarty-amd authored Oct 07, 2025

41f1cf38
[Model] Use `merge_by_field_config` for MM models (Ovis family) (#26308) · 08d26a1b
Isotr0py authored Oct 07, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
08d26a1b

Add topk logits torch op for DS3.2. (#25945) · e1098ced

Daniel Cámpora authored Oct 07, 2025

Signed-off-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
Signed-off-by: Daniel Cámpora <961215+dcampora@users.noreply.github.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>

e1098ced

Optimize KV cache distribution for asymmetric pipeline parallelism (#25164) · d100d78e
Grant Holmes (Ren) authored Oct 07, 2025
```
Signed-off-by: gholmes829 <g.holmes429@gmail.com>
```
d100d78e

[Model] Define merge_by_field_config MM interface (R-T) (#26260) · de342585

Ayush Satyam authored Oct 07, 2025


Signed-off-by: Ayush Satyam <ayushsatyam146@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

de342585

[responsesAPI][bugfix] serialize harmony messages (#26185) · 185d8ed4
Andrew Xia authored Oct 07, 2025
```
Signed-off-by: Andrew Xia <axia@meta.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
```
185d8ed4
[Deprecation] Deprecate `LLM.set_tokenizer` (#26333) · d9836d45
Cyrus Leung authored Oct 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
d9836d45

[Model] Define merge_by_field_config MM interface (U-Z) (#26261) · 5f7e8a91

Ayush Satyam authored Oct 07, 2025


Signed-off-by: Ayush Satyam <ayushsatyam146@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

5f7e8a91

[BUG] Fix file parsing for load_format runai_streamer_sharded (#26324) · 4dbdf4a2
ahao-anyscale authored Oct 06, 2025
```
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
```
4dbdf4a2
[UX] Support nested dicts in hf_overrides (#25727) · c6873c4e
Michael Goin authored Oct 06, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
c6873c4e

[Core] Simplify the Dp padding/should ubatch coordination logic (#25768) · 2111b464

Sage Moore authored Oct 06, 2025


Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>

2111b464

06 Oct, 2025 4 commits
- [Perf] Add decode full-graph support to FlashInfer-MLA backend (#26313) · f77df946
  Benjamin Chislett authored Oct 06, 2025
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
  f77df946
- [ROCm] Split AITER unified attention into its own backend (#25507) · f231e5bc
  Gregory Shtrasberg authored Oct 06, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  f231e5bc
- [Bugfix] Allow skipping MoE in NVFP4 (fix for MTP) (#25987) · 2161efe9
  Benjamin Chislett authored Oct 06, 2025
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
  2161efe9
- [BugFix] Pad input buffers in _dummy_run (#26209) · f23b4c04
  Varun Sundar Rabindranath authored Oct 06, 2025
```
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
  f23b4c04