Commits · d83f3f7cb37a0f1861f16c84d529abcd54889885 · OpenDAS / vllm_cscc

23 Sep, 2025 6 commits

[Perf] Increase default max splits for FA3 full cudagraphs (#25495) · e0b24ea0
Lucas Wilkinson authored Sep 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
e0b24ea0
Add `VLLM_NVTX_SCOPES_FOR_PROFILING=1` to enable `nvtx.annotate` scopes (#25501) · c85d75cf
Corey Lowman authored Sep 23, 2025
```
Signed-off-by: Corey Lowman <clowman1993@gmail.com>
```
c85d75cf
Add VLLM_ENABLE_INDUCTOR_MAX_AUTOTUNE & VLLM_ENABLE_INDUCTOR_COORDINA… (#25493) · eca7be90
rouchenzi authored Sep 23, 2025
```
Signed-off-by: rouchenzi <ruochenwen@gmail.com>
Signed-off-by: rouchenzi <40842833+rouchenzi@users.noreply.github.com>
```
eca7be90
Enable symmetric memory all reduce by default only enabling for TP (#25070) · 8bdd8b5c
Ilya Markov authored Sep 23, 2025
```
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
8bdd8b5c

[core] add nccl symmetric memory for all reduce (#24532) · 8c1c81a3

Amir Samani authored Sep 23, 2025


Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

8c1c81a3

[Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) · cc1dc7ed

Lucas Wilkinson authored Sep 23, 2025


Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

cc1dc7ed

22 Sep, 2025 4 commits

[Frontend] Responses API MCP tools for built in tools and to pass through headers (#24628) · 45d7d852

Alec S authored Sep 22, 2025


Signed-off-by: Alec Solder <alecs@fb.com>
Signed-off-by: Alec S <10566873+alecsolder@users.noreply.github.com>
Co-authored-by: Alec Solder <alecs@fb.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>

45d7d852

[torch.compile] Cleanup compilation tests and custom passes, add debug utils,... · d5e0fca2

Luka Govedič authored Sep 22, 2025


[torch.compile] Cleanup compilation tests and custom passes, add debug utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604) (#24542)
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

d5e0fca2

[Compiler] Disable Inductor standalone compile by default (#25391) · 1d7f95b8
ElizaWszola authored Sep 22, 2025
```
Signed-off-by: ElizaWszola <ewszola@redhat.com>
```
1d7f95b8

[CLI env var] Add VLLM_FLASH_ATTN_MAX_NUM_SPLITS_FOR_CUDA_GRAPH in env variables (#25274) · cfbee3d0

Daisy-Ma-coder authored Sep 22, 2025


Signed-off-by: qqma <qqma@amazon.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: qqma <qqma@amazon.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

cfbee3d0

21 Sep, 2025 1 commit
- Remove V0 attention backends (#25351) · bc6e542d
  Woosuk Kwon authored Sep 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  bc6e542d
20 Sep, 2025 1 commit
- [Bugfix] Remove VLLM_TEST_DYNAMO_FULLGRAPH_CAPTURE #2969 (#25090) · 3da17c2c
  Lucas Kabela authored Sep 19, 2025
```
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
```
  3da17c2c
19 Sep, 2025 1 commit
- allow disable flashinfer prefill (#25276) · ee7a66dd
  Lucia Fang authored Sep 19, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
```
  ee7a66dd
18 Sep, 2025 2 commits

Enable Allgather/ReduceScatter backend for NaiveAllToAll (#23964) · 2ea50e97

Shu Wang authored Sep 18, 2025


Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Shu Wang <shuw@nvidia.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

2ea50e97

[Kernels] Enable DeepGEMM by default (#24462) · 4ac510f4
bnellnm authored Sep 17, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
4ac510f4

17 Sep, 2025 2 commits
- [Bugfix] Refactor Flashinfer TRTLLM attention kernel selection logic (#24600) · e67a79db
  elvischenv authored Sep 18, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  e67a79db
- [UX] Enforce valid choices for envs like VLLM_ATTENTION_BACKEND, etc (#24761) · dd83a157
  Michael Goin authored Sep 16, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
```
  dd83a157
15 Sep, 2025 3 commits

[ci] fix wheel names for arm wheels (#24898) · fd2f1054
Simon Mo authored Sep 15, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
fd2f1054
Update num_tokens_across_dp to use nccl instead of gloo (#24105) · 49bfc538
Sage Moore authored Sep 15, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
49bfc538

Fp8 paged attention update (#22222) · 01413e0c

xiao-llm authored Sep 15, 2025


Signed-off-by: Xiao Yu <xiao.yu@amd.com>
Signed-off-by: xiao-llm <xiao.yu.dc@outlook.com>
Co-authored-by: Xiao Yu <xiao.yu@metamaterial.com>
Co-authored-by: Xiao Yu <xiao.yu@amd.com>
Co-authored-by: Bowen Bao <bowenbao@amd.com>

01413e0c

12 Sep, 2025 2 commits
- [Attention][FlashInfer] Enable FP8 FlashInfer (TRTLLM) MLA decode (#24705) · 7ba32aa6
  Matthew Bonanni authored Sep 12, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
```
  7ba32aa6
- [Core] Shared memory based object store for Multimodal data caching and IPC (#20452) · a5b84f1c
  dongluw authored Sep 12, 2025
```
Signed-off-by: donglu <donglu@cohere.com>
```
  a5b84f1c
11 Sep, 2025 4 commits

[Bugfix] Set `VLLM_ALLREDUCE_USE_SYMM_MEM` default to False (#24696) · 1ec20355
Wentao Ye authored Sep 11, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
1ec20355
[flashinfer] [kernel] support for fp8 kv cache for trtllm prefill attention (#24197) · e42af78b
Xiaozhu Meng authored Sep 11, 2025
```
Signed-off-by: Xiaozhu <mxz297@gmail.com>
```
e42af78b

[Kernel][B200] `mxfp4` fused cutlass moe (#23696) · 074854b2

Duncan Moss authored Sep 11, 2025


Signed-off-by: Duncan Moss <djm.moss@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

074854b2

[Kernels] Enable Torch Symmetric Memory All-Reduce By Default (#24111) · 1fdd5c42
Ilya Markov authored Sep 11, 2025
```
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
1fdd5c42

10 Sep, 2025 1 commit
- [Logging] allow config logging stream (#24336) · 2bef2d14
  Shiyan Deng authored Sep 10, 2025
```
Signed-off-by: Shiyan Deng <dsy842974287@meta.com>
```
  2bef2d14
09 Sep, 2025 2 commits

[Core] Use sha256 bytes instead of BlockHash to reduce GC overhead (#23673) · 82dfb12e
Zebing Lin authored Sep 09, 2025
```
Signed-off-by: linzebing <linzebing1995@gmail.com>
```
82dfb12e

[gpt-oss] Harmony changes with container tool support (#23386) · 170129eb

zhiweiz authored Sep 08, 2025


Signed-off-by: zhiweiz <zhiweiz@fb.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
Co-authored-by: zhiweiz <zhiweiz@fb.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>

170129eb

06 Sep, 2025 3 commits
- [V0 deprecation] Deprecate V0 Neuron backend (#21159) · 4172235a
  Woosuk Kwon authored Sep 06, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  4172235a
- break execute_model in gpu_model_runner into sub-functions for custom scopes (#24265) · 848562bd
  Bangsheng Tang authored Sep 06, 2025
```
Co-authored-by: Bangsheng Tang <bangsheng@meta.com>
```
  848562bd
- [Doc]: fix typos in Python comments (#24294) · 35bf1938
  Didier Durand authored Sep 06, 2025
```
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
```
  35bf1938
04 Sep, 2025 1 commit

[Attention] FlashAttn MLA (#14258) · 402759d4

Lucas Wilkinson authored Sep 04, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>

402759d4

28 Aug, 2025 1 commit

[ROCm][Aiter] Add triton fp8 bmm kernel for mla (#23264) · 04d1dd7f

Divakar Verma authored Aug 28, 2025


Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Co-authored-by: ShaoChunLee <Shao-Chun.Lee@amd.com>

04d1dd7f

27 Aug, 2025 2 commits

[Feature] Add `VLLM_DISABLE_PAD_FOR_CUDAGRAPH` to Avoid Hang Issue (#23595) · 321938e9

Wentao Ye authored Aug 27, 2025

Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

321938e9

[Feature] Add Hopper DeepGEMM E8M0 for DeepSeekV3.1 scale_fmt (#23666) · 3af47c3c

Wentao Ye authored Aug 27, 2025


Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>

3af47c3c

26 Aug, 2025 1 commit
- [Misc] Add override for allreduce fusion thresholds (#23639) · 7ea22e42
  nvjullin authored Aug 26, 2025
```
Signed-off-by: Julien Lin <jullin@nvidia.com>
```
  7ea22e42
25 Aug, 2025 1 commit

[Kernel] Add fused grouped_topk kernel for MoE (#23274) · 8a3cd90a

Xin Yang authored Aug 25, 2025


Signed-off-by: Xin Yang <xyangx@amazon.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

8a3cd90a

22 Aug, 2025 2 commits

[PERF] PyTorch Symmetric Memory All-Reduce (#20759) · 0313cf85

Ilya Markov authored Aug 22, 2025


Signed-off-by: ilmarkov <imarkov@redhat.com>
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

0313cf85

add an env var for path to pre-downloaded flashinfer cubin files (#22675) · da65bec3
Shiyan Deng authored Aug 22, 2025

da65bec3