Commits · c6187f55f7c4844ed9ff5630d41114cbe6fccb6b · OpenDAS / vllm_cscc

08 Oct, 2025 1 commit

[Feature] Enable E8M0 by Default on Hopper for DeepGEMM, 5% E2E throughput improvement (#26197) · f8607863

Wentao Ye authored Oct 08, 2025

Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

f8607863

07 Oct, 2025 1 commit

[Core] Simplify the Dp padding/should ubatch coordination logic (#25768) · 2111b464

Sage Moore authored Oct 06, 2025


Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>

2111b464

06 Oct, 2025 1 commit
- [ROCm] Split AITER unified attention into its own backend (#25507) · f231e5bc
  Gregory Shtrasberg authored Oct 06, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  f231e5bc
05 Oct, 2025 1 commit
- Convert formatting to use `ruff` instead of `yapf` + `isort` (#26247) · d6953beb
  Harry Mellor authored Oct 05, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6953beb
03 Oct, 2025 1 commit
- [Attention] Move Backend enum into registry (#25893) · 2aaa4238
  Matthew Bonanni authored Oct 02, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  2aaa4238
02 Oct, 2025 1 commit
- [Small] Prevent bypassing media domain restriction via HTTP redirects (#26035) · ad87ba92
  Chenheli Hua authored Oct 02, 2025
```
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
```
  ad87ba92
01 Oct, 2025 1 commit

[Misc]allow disable pynccl (#25421) · f48b6a03

Lucia Fang authored Sep 30, 2025


Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com>

f48b6a03

29 Sep, 2025 1 commit
- [NIXL] Increase default KV block eviction timeout on P (#25897) · 2e4fe48c
  Nicolò Lucchesi authored Sep 29, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  2e4fe48c
27 Sep, 2025 3 commits

[Core] GC Debug callback (#24829) · c216119d

Jialin Ouyang authored Sep 27, 2025


Signed-off-by: Jialin Ouyang <jialino@meta.com>
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Co-authored-by: Jialin Ouyang <jialino@meta.com>

c216119d

[torch.compile]: Add VLLM_DEBUG_DUMP_PATH environment variable (#25651) · c0ec8183

Jiangyun Zhu authored Sep 28, 2025


Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

c0ec8183

[env] default nixl side port conflicts with kv-event zmq port (#25056) · 49996cd5
Peter Pan authored Sep 27, 2025
```
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
```
49996cd5

26 Sep, 2025 1 commit

Llamas 3.1 405B fp4 changes upstreaming from 355_wip (#25135) · 53a30845

Aleksandr Malyshev authored Sep 25, 2025


Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Doug Lehr <douglehr@amd.com>

53a30845

25 Sep, 2025 1 commit
- Enable Fbgemm NVFP4 on Dense models (#25609) · 90b139cf
  Saman A. Pour authored Sep 24, 2025
```
Signed-off-by: Saman Keon <samanamp@outlook.com>
```
  90b139cf
24 Sep, 2025 3 commits

feat: BF16 FlashInfer Fused Cutlass MOE for Hopper and Blackwell Expert Parallel (#25503) · 6160ba41
Duncan Moss authored Sep 24, 2025
```
Signed-off-by: Duncan Moss <djm.moss@gmail.com>
```
6160ba41

Support mnnvl all2allv from Flashinfer (#21003) · 54e42b72

Shu Wang authored Sep 24, 2025


Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

54e42b72

[Bugfix] add cache model when from object storage get model (#24764) · 2dda3e35
rongfu.leng authored Sep 25, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
2dda3e35

23 Sep, 2025 6 commits

[Perf] Increase default max splits for FA3 full cudagraphs (#25495) · e0b24ea0
Lucas Wilkinson authored Sep 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
e0b24ea0
Add `VLLM_NVTX_SCOPES_FOR_PROFILING=1` to enable `nvtx.annotate` scopes (#25501) · c85d75cf
Corey Lowman authored Sep 23, 2025
```
Signed-off-by: Corey Lowman <clowman1993@gmail.com>
```
c85d75cf
Add VLLM_ENABLE_INDUCTOR_MAX_AUTOTUNE & VLLM_ENABLE_INDUCTOR_COORDINA… (#25493) · eca7be90
rouchenzi authored Sep 23, 2025
```
Signed-off-by: rouchenzi <ruochenwen@gmail.com>
Signed-off-by: rouchenzi <40842833+rouchenzi@users.noreply.github.com>
```
eca7be90
Enable symmetric memory all reduce by default only enabling for TP (#25070) · 8bdd8b5c
Ilya Markov authored Sep 23, 2025
```
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
8bdd8b5c

[core] add nccl symmetric memory for all reduce (#24532) · 8c1c81a3

Amir Samani authored Sep 23, 2025


Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

8c1c81a3

[Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) · cc1dc7ed

Lucas Wilkinson authored Sep 23, 2025


Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

cc1dc7ed

22 Sep, 2025 4 commits

[Frontend] Responses API MCP tools for built in tools and to pass through headers (#24628) · 45d7d852

Alec S authored Sep 22, 2025


Signed-off-by: Alec Solder <alecs@fb.com>
Signed-off-by: Alec S <10566873+alecsolder@users.noreply.github.com>
Co-authored-by: Alec Solder <alecs@fb.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>

45d7d852

[torch.compile] Cleanup compilation tests and custom passes, add debug utils,... · d5e0fca2

Luka Govedič authored Sep 22, 2025


[torch.compile] Cleanup compilation tests and custom passes, add debug utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604) (#24542)
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

d5e0fca2

[Compiler] Disable Inductor standalone compile by default (#25391) · 1d7f95b8
ElizaWszola authored Sep 22, 2025
```
Signed-off-by: ElizaWszola <ewszola@redhat.com>
```
1d7f95b8

[CLI env var] Add VLLM_FLASH_ATTN_MAX_NUM_SPLITS_FOR_CUDA_GRAPH in env variables (#25274) · cfbee3d0

Daisy-Ma-coder authored Sep 22, 2025


Signed-off-by: qqma <qqma@amazon.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: qqma <qqma@amazon.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

cfbee3d0

21 Sep, 2025 1 commit
- Remove V0 attention backends (#25351) · bc6e542d
  Woosuk Kwon authored Sep 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  bc6e542d
20 Sep, 2025 1 commit
- [Bugfix] Remove VLLM_TEST_DYNAMO_FULLGRAPH_CAPTURE #2969 (#25090) · 3da17c2c
  Lucas Kabela authored Sep 19, 2025
```
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
```
  3da17c2c
19 Sep, 2025 1 commit
- allow disable flashinfer prefill (#25276) · ee7a66dd
  Lucia Fang authored Sep 19, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
```
  ee7a66dd
18 Sep, 2025 2 commits

Enable Allgather/ReduceScatter backend for NaiveAllToAll (#23964) · 2ea50e97

Shu Wang authored Sep 18, 2025


Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Shu Wang <shuw@nvidia.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

2ea50e97

[Kernels] Enable DeepGEMM by default (#24462) · 4ac510f4
bnellnm authored Sep 17, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
4ac510f4

17 Sep, 2025 2 commits
- [Bugfix] Refactor Flashinfer TRTLLM attention kernel selection logic (#24600) · e67a79db
  elvischenv authored Sep 18, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  e67a79db
- [UX] Enforce valid choices for envs like VLLM_ATTENTION_BACKEND, etc (#24761) · dd83a157
  Michael Goin authored Sep 16, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
```
  dd83a157
15 Sep, 2025 3 commits

[ci] fix wheel names for arm wheels (#24898) · fd2f1054
Simon Mo authored Sep 15, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
fd2f1054
Update num_tokens_across_dp to use nccl instead of gloo (#24105) · 49bfc538
Sage Moore authored Sep 15, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
49bfc538

Fp8 paged attention update (#22222) · 01413e0c

xiao-llm authored Sep 15, 2025


Signed-off-by: Xiao Yu <xiao.yu@amd.com>
Signed-off-by: xiao-llm <xiao.yu.dc@outlook.com>
Co-authored-by: Xiao Yu <xiao.yu@metamaterial.com>
Co-authored-by: Xiao Yu <xiao.yu@amd.com>
Co-authored-by: Bowen Bao <bowenbao@amd.com>

01413e0c

12 Sep, 2025 2 commits
- [Attention][FlashInfer] Enable FP8 FlashInfer (TRTLLM) MLA decode (#24705) · 7ba32aa6
  Matthew Bonanni authored Sep 12, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
```
  7ba32aa6
- [Core] Shared memory based object store for Multimodal data caching and IPC (#20452) · a5b84f1c
  dongluw authored Sep 12, 2025
```
Signed-off-by: donglu <donglu@cohere.com>
```
  a5b84f1c
11 Sep, 2025 2 commits
- [Bugfix] Set `VLLM_ALLREDUCE_USE_SYMM_MEM` default to False (#24696) · 1ec20355
  Wentao Ye authored Sep 11, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  1ec20355
- [flashinfer] [kernel] support for fp8 kv cache for trtllm prefill attention (#24197) · e42af78b
  Xiaozhu Meng authored Sep 11, 2025
```
Signed-off-by: Xiaozhu <mxz297@gmail.com>
```
  e42af78b