Commits · de92d916fe8a897b00a8adb0aab9ed9ec99f2b6c · OpenDAS / vllm_cscc

15 Oct, 2025 1 commit

[NVIDIA] Add support for cudnn fp4 gemm via flashinfer (#26107) · de92d916

Kaixi Hou authored Oct 15, 2025


Signed-off-by: kaixih <kaixih@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>

de92d916

14 Oct, 2025 3 commits
- [Perf] Cache vllm.env.__getattr__ result to avoid recomputation (#26146) · 380f1752
  Jialin Ouyang authored Oct 14, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  380f1752
- [Config] Remove Unused Environment Variable `VLLM_DISABLE_PAD_FOR_CUDAGRAPH` (#26743) · 6d87a283
  Wentao Ye authored Oct 14, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  6d87a283
- [Easy] Fix env type check errors from VLLM_DEBUG_LOG_API_SERVER_RESPONSE (#26742) · cfded807
  Jialin Ouyang authored Oct 13, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  cfded807
13 Oct, 2025 1 commit

[UX] Speedup DeepGEMM warmup with heuristics (#25619) · 0d21b9b5

Michael Goin authored Oct 13, 2025

Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>

0d21b9b5

12 Oct, 2025 1 commit
- Update `Optional[x]` -> `x | None` and `Union[x, y]` to `x | y` (#26633) · 8fcaaf6a
  Harry Mellor authored Oct 12, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8fcaaf6a
10 Oct, 2025 2 commits
- AOT Compilation for torch.compile (Bundled) (#24274) · eef921f4
  Zhengxu Chen authored Oct 10, 2025
```
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
```
  eef921f4
- [DP][ray] Support different VLLM_RAY_DP_PACK_STRATEGY (#23849) · 757fa4a4
  Rui Qiao authored Oct 09, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  757fa4a4
08 Oct, 2025 1 commit

[Feature] Enable E8M0 by Default on Hopper for DeepGEMM, 5% E2E throughput improvement (#26197) · f8607863

Wentao Ye authored Oct 08, 2025

Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

f8607863

07 Oct, 2025 1 commit

[Core] Simplify the Dp padding/should ubatch coordination logic (#25768) · 2111b464

Sage Moore authored Oct 06, 2025


Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>

2111b464

06 Oct, 2025 1 commit
- [ROCm] Split AITER unified attention into its own backend (#25507) · f231e5bc
  Gregory Shtrasberg authored Oct 06, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  f231e5bc
05 Oct, 2025 1 commit
- Convert formatting to use `ruff` instead of `yapf` + `isort` (#26247) · d6953beb
  Harry Mellor authored Oct 05, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6953beb
03 Oct, 2025 1 commit
- [Attention] Move Backend enum into registry (#25893) · 2aaa4238
  Matthew Bonanni authored Oct 02, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  2aaa4238
02 Oct, 2025 1 commit
- [Small] Prevent bypassing media domain restriction via HTTP redirects (#26035) · ad87ba92
  Chenheli Hua authored Oct 02, 2025
```
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
```
  ad87ba92
01 Oct, 2025 1 commit

[Misc]allow disable pynccl (#25421) · f48b6a03

Lucia Fang authored Sep 30, 2025


Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com>

f48b6a03

29 Sep, 2025 1 commit
- [NIXL] Increase default KV block eviction timeout on P (#25897) · 2e4fe48c
  Nicolò Lucchesi authored Sep 29, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  2e4fe48c
27 Sep, 2025 3 commits

[Core] GC Debug callback (#24829) · c216119d

Jialin Ouyang authored Sep 27, 2025


Signed-off-by: Jialin Ouyang <jialino@meta.com>
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Co-authored-by: Jialin Ouyang <jialino@meta.com>

c216119d

[torch.compile]: Add VLLM_DEBUG_DUMP_PATH environment variable (#25651) · c0ec8183

Jiangyun Zhu authored Sep 28, 2025


Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

c0ec8183

[env] default nixl side port conflicts with kv-event zmq port (#25056) · 49996cd5
Peter Pan authored Sep 27, 2025
```
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
```
49996cd5

26 Sep, 2025 1 commit

Llamas 3.1 405B fp4 changes upstreaming from 355_wip (#25135) · 53a30845

Aleksandr Malyshev authored Sep 25, 2025


Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Doug Lehr <douglehr@amd.com>

53a30845

25 Sep, 2025 1 commit
- Enable Fbgemm NVFP4 on Dense models (#25609) · 90b139cf
  Saman A. Pour authored Sep 24, 2025
```
Signed-off-by: Saman Keon <samanamp@outlook.com>
```
  90b139cf
24 Sep, 2025 3 commits

feat: BF16 FlashInfer Fused Cutlass MOE for Hopper and Blackwell Expert Parallel (#25503) · 6160ba41
Duncan Moss authored Sep 24, 2025
```
Signed-off-by: Duncan Moss <djm.moss@gmail.com>
```
6160ba41

Support mnnvl all2allv from Flashinfer (#21003) · 54e42b72

Shu Wang authored Sep 24, 2025


Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

54e42b72

[Bugfix] add cache model when from object storage get model (#24764) · 2dda3e35
rongfu.leng authored Sep 25, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
2dda3e35

23 Sep, 2025 6 commits

[Perf] Increase default max splits for FA3 full cudagraphs (#25495) · e0b24ea0
Lucas Wilkinson authored Sep 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
e0b24ea0
Add `VLLM_NVTX_SCOPES_FOR_PROFILING=1` to enable `nvtx.annotate` scopes (#25501) · c85d75cf
Corey Lowman authored Sep 23, 2025
```
Signed-off-by: Corey Lowman <clowman1993@gmail.com>
```
c85d75cf
Add VLLM_ENABLE_INDUCTOR_MAX_AUTOTUNE & VLLM_ENABLE_INDUCTOR_COORDINA… (#25493) · eca7be90
rouchenzi authored Sep 23, 2025
```
Signed-off-by: rouchenzi <ruochenwen@gmail.com>
Signed-off-by: rouchenzi <40842833+rouchenzi@users.noreply.github.com>
```
eca7be90
Enable symmetric memory all reduce by default only enabling for TP (#25070) · 8bdd8b5c
Ilya Markov authored Sep 23, 2025
```
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
8bdd8b5c

[core] add nccl symmetric memory for all reduce (#24532) · 8c1c81a3

Amir Samani authored Sep 23, 2025


Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

8c1c81a3

[Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) · cc1dc7ed

Lucas Wilkinson authored Sep 23, 2025


Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

cc1dc7ed

22 Sep, 2025 4 commits

[Frontend] Responses API MCP tools for built in tools and to pass through headers (#24628) · 45d7d852

Alec S authored Sep 22, 2025


Signed-off-by: Alec Solder <alecs@fb.com>
Signed-off-by: Alec S <10566873+alecsolder@users.noreply.github.com>
Co-authored-by: Alec Solder <alecs@fb.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>

45d7d852

[torch.compile] Cleanup compilation tests and custom passes, add debug utils,... · d5e0fca2

Luka Govedič authored Sep 22, 2025


[torch.compile] Cleanup compilation tests and custom passes, add debug utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604) (#24542)
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

d5e0fca2

[Compiler] Disable Inductor standalone compile by default (#25391) · 1d7f95b8
ElizaWszola authored Sep 22, 2025
```
Signed-off-by: ElizaWszola <ewszola@redhat.com>
```
1d7f95b8

[CLI env var] Add VLLM_FLASH_ATTN_MAX_NUM_SPLITS_FOR_CUDA_GRAPH in env variables (#25274) · cfbee3d0

Daisy-Ma-coder authored Sep 22, 2025


Signed-off-by: qqma <qqma@amazon.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: qqma <qqma@amazon.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

cfbee3d0

21 Sep, 2025 1 commit
- Remove V0 attention backends (#25351) · bc6e542d
  Woosuk Kwon authored Sep 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  bc6e542d
20 Sep, 2025 1 commit
- [Bugfix] Remove VLLM_TEST_DYNAMO_FULLGRAPH_CAPTURE #2969 (#25090) · 3da17c2c
  Lucas Kabela authored Sep 19, 2025
```
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
```
  3da17c2c
19 Sep, 2025 1 commit
- allow disable flashinfer prefill (#25276) · ee7a66dd
  Lucia Fang authored Sep 19, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
```
  ee7a66dd
18 Sep, 2025 2 commits

Enable Allgather/ReduceScatter backend for NaiveAllToAll (#23964) · 2ea50e97

Shu Wang authored Sep 18, 2025


Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Shu Wang <shuw@nvidia.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

2ea50e97

[Kernels] Enable DeepGEMM by default (#24462) · 4ac510f4
bnellnm authored Sep 17, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
4ac510f4

17 Sep, 2025 1 commit

[Bugfix] Refactor Flashinfer TRTLLM attention kernel selection logic (#24600) · e67a79db

elvischenv authored Sep 18, 2025


Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

e67a79db