Commits · d5e0fca264f7d277ed372f7c075827ed9a0c5e7e · OpenDAS / vllm_cscc

22 Sep, 2025 2 commits

[torch.compile] Cleanup compilation tests and custom passes, add debug utils,... · d5e0fca2

Luka Govedič authored Sep 22, 2025


[torch.compile] Cleanup compilation tests and custom passes, add debug utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604) (#24542)
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

d5e0fca2

[CLI env var] Add VLLM_FLASH_ATTN_MAX_NUM_SPLITS_FOR_CUDA_GRAPH in env variables (#25274) · cfbee3d0

Daisy-Ma-coder authored Sep 22, 2025


Signed-off-by: qqma <qqma@amazon.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: qqma <qqma@amazon.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

cfbee3d0

21 Sep, 2025 2 commits
- Remove V0 attention backends (#25351) · bc6e542d
  Woosuk Kwon authored Sep 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  bc6e542d
- [V0 Deprecation] Remove LLMEngine (#25033) · 52c2a8d4
  Woosuk Kwon authored Sep 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  52c2a8d4
20 Sep, 2025 2 commits

[torch.compile] CUDAGraph Inductor partition integration (#24281) · 8945b001

Boyuan Feng authored Sep 19, 2025


Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Boyuan Feng <fby.1994@gmail.com>
Signed-off-by: boyuanfeng <boyuan@meta.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

8945b001

[Bugfix] Remove VLLM_TEST_DYNAMO_FULLGRAPH_CAPTURE #2969 (#25090) · 3da17c2c
Lucas Kabela authored Sep 19, 2025
```
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
```
3da17c2c

17 Sep, 2025 1 commit

[Bugfix] Fix accuracy issue for silu_mul + nvfp4 quant fusion kernel (#24833) · e6585ddb

elvischenv authored Sep 18, 2025


Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

e6585ddb

15 Sep, 2025 1 commit
- [Compile] Fix noop_elimination pass and add tests for noop_elimination (#24880) · 5bcc153d
  Jiangyun Zhu authored Sep 16, 2025
```
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
```
  5bcc153d
11 Sep, 2025 1 commit
- fix some typos (#24616) · e26fef83
  co63oc authored Sep 12, 2025
```
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
```
  e26fef83
10 Sep, 2025 1 commit

[torch.compile][ROCm][V1] Enable attention output FP8 fusion for V1 attention backends (#19767) · 9a161307

Gregory Shtrasberg authored Sep 10, 2025


Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

9a161307

09 Sep, 2025 1 commit
- [CI] execute all piecewise compilation tests together (#24502) · b8a93076
  Jiangyun Zhu authored Sep 10, 2025
```
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
```
  b8a93076
08 Sep, 2025 2 commits
- [Attention] FlashAttention MLA cudagraph support (#23958) · 620db1fc
  Matthew Bonanni authored Sep 08, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
```
  620db1fc
- [CI] Enable encoder model compilation test (#24442) · 7be141b2
  Jiangyun Zhu authored Sep 09, 2025
```
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
```
  7be141b2
06 Sep, 2025 1 commit
- [Bugfix] Fix unstable silu_mul+nvfp4 quant fusion test (#24370) · e68dc2f0
  elvischenv authored Sep 07, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
```
  e68dc2f0
05 Sep, 2025 1 commit
- [Bugfix] Fix silu_mul+quant fusion test (#24341) · eedb2a2a
  elvischenv authored Sep 06, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
```
  eedb2a2a
04 Sep, 2025 1 commit

[Misc] Removed force_fp8_e4m3fnuz from FP8LinearOp (#23725) · 37241077

nvjullin authored Sep 04, 2025

Signed-off-by: Julien Lin <jullin@nvidia.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

37241077

03 Sep, 2025 1 commit
- fix some typos (#24071) · 1bd007f2
  co63oc authored Sep 03, 2025
```
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
```
  1bd007f2
28 Aug, 2025 1 commit

[NVIDIA] Support SiluMul + NVFP4 quant fusion (#23671) · 16a45b3a

elvischenv authored Aug 29, 2025


Signed-off-by: jindih <jindih@nvidia.com>
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: jindih <jindih@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedic <lgovedic@redhat.com>

16a45b3a

26 Aug, 2025 1 commit
- [Kernel] Added flashinfer fp8 per-tensor gemms (#22895) · f66673a3
  nvjullin authored Aug 26, 2025
```
Signed-off-by: Julien Lin <jullin@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  f66673a3
22 Aug, 2025 1 commit

[NVIDIA][torch.compile] Support Flashinfer TRTLLM FP8-q/kv NVFP4-out Attention Kernel (#22703) · 24d0c9e6

elvischenv authored Aug 23, 2025


Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

24d0c9e6

20 Aug, 2025 3 commits
- [Kernel/Quant] Remove the original marlin format and qqq (#23204) · 0cdbf5e6
  Michael Goin authored Aug 20, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  0cdbf5e6
- [torch.compile] Support conditional torch.compile per module (#22269) · dfd23820
  Yong Hoon Shin authored Aug 20, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  dfd23820
- [CI Perf] Only test bfloat16 for tests/compile/test_fusion_all_reduce.py (#23132) · d46d417b
  Michael Goin authored Aug 19, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  d46d417b
19 Aug, 2025 2 commits

[NVIDIA] Support Flashinfer TRTLLM FP8-q/kv/out Attention Kernel (#21716) · 03752dba

elvischenv authored Aug 19, 2025

Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

03752dba

[V0 Deprecation] Remove V0 FlashInfer attention backend (#22776) · 14006840
Woosuk Kwon authored Aug 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
14006840

16 Aug, 2025 1 commit

[Kernel/Quant] Remove AQLM (#22943) · 4fc722ec

Michael Goin authored Aug 16, 2025


Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

4fc722ec

15 Aug, 2025 2 commits

[Core] Allow full cudagraph with separate attention routines and orthogonal to... · 74f441f4

fhl2000 authored Aug 15, 2025


[Core] Allow full cudagraph with separate attention routines and orthogonal to compilation, add support for FA2 and FlashInfer (#20059)
Signed-off-by: fhl <2410591650@qq.com>
Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>

74f441f4

[Feature] Full Cuda Graph Support for Cutlass MLA and 6% E2E Throughput Improvement (#22763) · 5c3fbfe4
Wentao Ye authored Aug 15, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
5c3fbfe4

01 Aug, 2025 1 commit
- [CI] Initial tests for SM100 Blackwell runner (#21877) · 88faa466
  Michael Goin authored Aug 01, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  88faa466
31 Jul, 2025 1 commit

Add FlashInfer allreduce RMSNorm Quant fusion (#21069) · 6e672daf

Ilya Markov authored Jul 31, 2025


Signed-off-by: ilmarkov <imarkov@redhat.com>
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>

6e672daf

30 Jul, 2025 1 commit
- [Feature] Add async tensor parallelism for scaled mm (#20155) · 287f527f
  cascade authored Jul 30, 2025
```
Signed-off-by: cascade812 <cascade812@outlook.com>
```
  287f527f
29 Jul, 2025 1 commit
- [Bugfix] VLLM_V1 supports passing other compilation levels (#19340) · 04e38500
  Richard Zou authored Jul 29, 2025
```
Signed-off-by: Richard Zou <zou3519@gmail.com>
```
  04e38500
28 Jul, 2025 1 commit

[Deprecation][2/N] Replace `--task` with `--runner` and `--convert` (#21470) · 86ae693f

Cyrus Leung authored Jul 28, 2025

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

86ae693f

23 Jul, 2025 1 commit
- Add test case for compiling multiple graphs (#21044) · 4ac7713e
  Yong Hoon Shin authored Jul 23, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  4ac7713e
15 Jul, 2025 2 commits
- [Misc] Refactor AllReduceFusionPass. Remove parameter (#20918) · 37a7d5d7
  Ilya Markov authored Jul 15, 2025
```
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
```
  37a7d5d7
- [BugFix] VLLM_DISABLE_COMPILE_CACHE=1 should disable all reads and writes from the cache (#20942) · ba8c3000
  Richard Zou authored Jul 14, 2025
```
Signed-off-by: Richard Zou <zou3519@gmail.com>
```
  ba8c3000
12 Jul, 2025 1 commit
- Integration SM100 FlashInfer fused allreduce RMSNorm (#20691) · fc0f41d1
  Ilya Markov authored Jul 12, 2025
```
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
```
  fc0f41d1
11 Jul, 2025 2 commits
- [Bugfix] Upgrade depyf to 0.19 and streamline custom pass logging (#20777) · 762be26a
  Luka Govedič authored Jul 11, 2025
```
Signed-off-by: Luka Govedic <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
```
  762be26a
- [Perf][fp8] Use CustomOp abstraction for fp8 quant for better perf (#19830) · 31d5c179
  Luka Govedič authored Jul 11, 2025
```
Signed-off-by: Luka Govedic <lgovedic@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  31d5c179
04 Jul, 2025 1 commit
- [Misc] Add SPDX-FileCopyrightText (#20428) · 1caca5a5
  Jee Jee Li authored Jul 04, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  1caca5a5