Commits · fafe76b4af175d48b9b69204c995ee167b46aca1 · OpenDAS / vllm_cscc

23 Mar, 2026 1 commit

[Async][Spec Decoding] Zero-bubble async scheduling + spec decoding (#32951) · fafe76b4

Matthew Bonanni authored Mar 23, 2026


Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Co-authored-by: zhrrr <43847754+izhuhaoran@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>

fafe76b4

16 Mar, 2026 1 commit
- [Spec Decode] Update extract_hidden_states to use deferred kv_connector clear (#37013) · 04bf5a35
  Fynn Schmitt-Ulms authored Mar 16, 2026
  
  04bf5a35
12 Mar, 2026 1 commit
- [Hardware] Replace torch.cuda.device_count/current_device/set_device API (#36145) · 53ec16a7
  Kunshang Ji authored Mar 12, 2026
```
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  53ec16a7
11 Mar, 2026 1 commit
- feat(spec_decode): fuse EAGLE step slot mapping and metadata updates (#33503) · 4aaaf8c8
  Sladyn authored Mar 10, 2026
```
Signed-off-by: sladynnunes <snunes@usc.edu>
```
  4aaaf8c8
09 Mar, 2026 1 commit
- Reapply [Attention] Refactor `check_and_update_config` (#35122) · 77a73458
  Matthew Bonanni authored Mar 09, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  77a73458
05 Mar, 2026 1 commit
- [Bugfix] Fix block_size for hybrid model MTP (#36036) · 57c629e9
  Benjamin Chislett authored Mar 05, 2026
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
  57c629e9
03 Mar, 2026 1 commit

[BugFix] Add support for MTP num_speculative_tokens > 1 with sparse MLA (#34552) · 28ef9ba3

Lucas Wilkinson authored Mar 03, 2026


Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>

28ef9ba3

02 Mar, 2026 1 commit
- [Spec Decode] Add hidden states extraction system (#33736) · 9433acb8
  Fynn Schmitt-Ulms authored Mar 02, 2026
```
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
```
  9433acb8
21 Feb, 2026 1 commit
- [ROCm][CI] Fix spec decode logprobs flakiness and parametrize tree attention backends (#34599) · 54254f7a
  Andreas Karatzas authored Feb 20, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
```
  54254f7a
20 Feb, 2026 1 commit
- [CI] Revert PRs 34818 and 33600 (#34979) · aaefc58e
  Lucas Wilkinson authored Feb 20, 2026
  
  aaefc58e
19 Feb, 2026 1 commit

[Bugfix] Fix Basic Models Test (#34818) · 662205d3

Matthew Bonanni authored Feb 19, 2026


Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

662205d3

05 Feb, 2026 1 commit
- [Spec Decode] Unified Parallel Drafting (#32887) · af3162d3
  Benjamin Chislett authored Feb 05, 2026
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
  af3162d3
02 Feb, 2026 1 commit
- move spec decode slow test to test_areas.yaml (#33365) · d95b4be4
  shanjiaz authored Feb 02, 2026
```
Signed-off-by: shanjiaz <zsjwpianpian@gmail.com>
```
  d95b4be4
29 Jan, 2026 1 commit
- Adding optional speculator tests for larger models (#32943) · 5eeba80c
  shanjiaz authored Jan 29, 2026
```
Signed-off-by: shanjiaz <zsjwpianpian@gmail.com>
```
  5eeba80c
24 Jan, 2026 1 commit

[Performance] Split FlashAttn attention and cache update (#25954) · a28b94e6

ElizaWszola authored Jan 24, 2026


Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Luka Govedič <luka.govedic@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <luka.govedic@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>

a28b94e6

22 Jan, 2026 3 commits
- [ROCm][CI][Docs] Add comment explaining TRITON_ATTN fallback for ROCm (#32835) · a8102998
  Andreas Karatzas authored Jan 22, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
```
  a8102998
- [ROCm][CI] fix get_valid_backends (#32787) · 49d96538
  Divakar Verma authored Jan 21, 2026
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  49d96538
- [Deprecation] Remove deprecated environment variables (#32812) · 6437ff1f
  Wentao Ye authored Jan 21, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  6437ff1f
20 Jan, 2026 1 commit
- Test: added acceptance length tests (#32030) · f0feb1cf
  Rahul Tuli authored Jan 21, 2026
```
Signed-off-by: rahul-tuli <rtuli@redhat.com>
```
  f0feb1cf
12 Jan, 2026 1 commit
- [3/N][Attention] Move AttentionMetadata-related code from utils.py to backend.py (#32054) · 20228cb8
  Matthew Bonanni authored Jan 12, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  20228cb8
09 Jan, 2026 2 commits
- [1/N][Attention] Restructure attention: move files (#31916) · 2612ba92
  Matthew Bonanni authored Jan 09, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  2612ba92
- [Cleanup] Remove obsolete spec decoding compatibility logic (#32003) · 29ce4822
  Nick Hill authored Jan 08, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
  29ce4822
22 Dec, 2025 1 commit
- [SpecDecode] Simplified alternative padded-speculation acceptance rate fix (#29845) · de717476
  Lucas Wilkinson authored Dec 22, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  de717476
17 Dec, 2025 1 commit
- [Attention] Update tests to remove deprecated env vars (#30563) · 7eb6cb6c
  Matthew Bonanni authored Dec 17, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  7eb6cb6c
10 Dec, 2025 1 commit

[Attention] Make seq_lens_cpu optional in CommonAttentionMetadata to enable... · abe93bce

Lucas Wilkinson authored Dec 09, 2025


[Attention] Make seq_lens_cpu optional in CommonAttentionMetadata to enable true async spec-decode (#29624)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>

abe93bce

09 Dec, 2025 1 commit
- [CI] Fix Flaky test_eagle_max_len Test (#30306) · aeb82b19
  Micah Williamson authored Dec 09, 2025
```
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
```
  aeb82b19
08 Dec, 2025 1 commit
- [ROCm][CI] Fix test_max_len.py for Rocm (#29916) · 6af70e11
  Charlie Fu authored Dec 08, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
Signed-off-by: Charlie Fu <Charlie.Fu@amd.com>
```
  6af70e11
07 Dec, 2025 2 commits
- Revert "[Renderer] Separate out `RendererConfig` from `ModelConfig` (#30145)" (#30199) · e83b7e37
  Cyrus Leung authored Dec 07, 2025
  
  e83b7e37
- [Renderer] Separate out `RendererConfig` from `ModelConfig` (#30145) · 27f4c2fd
  Cyrus Leung authored Dec 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  27f4c2fd
04 Dec, 2025 1 commit

[Rocm][CI] Fix test_speculator_eagle3 by skipping the CompressedTensorw4a16 Model (#30001) · 9aa33a74

Charlie Fu authored Dec 04, 2025


Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: Alexei-V-Ivanov-AMD <156011006+Alexei-V-Ivanov-AMD@users.noreply.github.com>

9aa33a74

02 Dec, 2025 2 commits
- Remove default values from `InitVar`s so that they're not stored (#29859) · 951445a5
  Harry Mellor authored Dec 02, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  951445a5
- [BugFix] Fix index error in ngram_proposer (#29779) · 81fe3f82
  usberkeley authored Dec 02, 2025
```
Signed-off-by: Bradley <bradley.b.pitt@gmail.com>
```
  81fe3f82
28 Nov, 2025 1 commit
- [Perf] Optimize EAGLE prepare_inputs_padded with triton kernels (#28597) · 1986de13
  Benjamin Chislett authored Nov 28, 2025
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
```
  1986de13
23 Nov, 2025 1 commit
- [CI/Build][AMD] Add check for flash_att_varlen_func to test_tree_attention.py (#29252) · 3999442f
  rasmith authored Nov 22, 2025
```
Signed-off-by: Randall Smith <ransmith@amd.com>
Co-authored-by: Randall Smith <ransmith@amd.com>
```
  3999442f
21 Nov, 2025 1 commit
- Revert "[Redo] #26368 (#28771)" (#29121) · 30b9c677
  Jialin Ouyang authored Nov 20, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  30b9c677
15 Nov, 2025 4 commits
- Add support for Eagle with separate lm-head and embed_tokens layers (#28549) · e439c784
  Eldar Kurtić authored Nov 15, 2025
```
Signed-off-by: Eldar Kurtic <8884008+eldarkurtic@users.noreply.github.com>
```
  e439c784
- [Redo] #26368 (#28771) · 98b4d389
  Cyrus Leung authored Nov 15, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  98b4d389
- Revert "[Core] Performance: Use list[np.ndarray] instead of list[list… (#28773) · ac86bff8
  Nick Hill authored Nov 14, 2025
  
  ac86bff8
- [Core] Performance: Use list[np.ndarray] instead of list[list[int]] for output... · 186352b2
  Jialin Ouyang authored Nov 14, 2025
```
[Core] Performance: Use list[np.ndarray] instead of list[list[int]] for output tokens for GC optimization (#26368)
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  186352b2
11 Nov, 2025 1 commit

[Attention] Refactor CUDA attention backend selection logic (#24794) · b30dfa03

Matthew Bonanni authored Nov 11, 2025


Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

b30dfa03