Commits · 54e42b72dbf7ec259de1c28e66ea9575fe594685 · OpenDAS / vllm_cscc

24 Sep, 2025 3 commits

Support mnnvl all2allv from Flashinfer (#21003) · 54e42b72

Shu Wang authored Sep 24, 2025


Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

54e42b72

[Misc] Improve type annotations for jsontree (#25577) · 9313be50
Cyrus Leung authored Sep 24, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
9313be50

[Spec Decode] Enable FlashInfer Spec Decoding (#25196) · c30b405b

Benjamin Chislett authored Sep 23, 2025


Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: lhsjohn <huashuoli@tencent.com>

c30b405b

23 Sep, 2025 4 commits

Remove redundant mutates_args and dispatch_key for direct_register_custom_op (#25512) · 7361ab37
Michael Goin authored Sep 23, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
7361ab37

[Performance] Move apply_w8a8_block_fp8_linear to an op class (#24666) · 63400259

ElizaWszola authored Sep 23, 2025


Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: ElizaWszola <elizaw.9289@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>

63400259

[core] add nccl symmetric memory for all reduce (#24532) · 8c1c81a3

Amir Samani authored Sep 23, 2025


Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

8c1c81a3

[Perf] Apply torch.compile for `per_block_cast_to_fp8` (#24611) · 9949aa2e
Wentao Ye authored Sep 22, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
9949aa2e

22 Sep, 2025 3 commits

[torch.compile] Cleanup compilation tests and custom passes, add debug utils,... · d5e0fca2

Luka Govedič authored Sep 22, 2025


[torch.compile] Cleanup compilation tests and custom passes, add debug utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604) (#24542)
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

d5e0fca2

Make `mypy` behave like a proper pre-commit hook (#25313) · 3d2c56b7

Harry Mellor authored Sep 22, 2025


Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

3d2c56b7

[Misc] Remove unused encoder-decoder error strings (#25374) · 417a164a
Cyrus Leung authored Sep 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
417a164a

21 Sep, 2025 1 commit
- Remove V0 attention backends (#25351) · bc6e542d
  Woosuk Kwon authored Sep 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  bc6e542d
20 Sep, 2025 2 commits
- [Docs] Fix warnings in mkdocs build (continued) (#25042) · 032d661d
  Wenlong Wang authored Sep 20, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  032d661d
- [BugFix] Exclude self when checking for port collision (#25286) · b7f186bb
  Nick Hill authored Sep 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  b7f186bb
19 Sep, 2025 1 commit

[CORE] Prompt Embeddings Support for v1 Engine (#24278) · 9a4600e4

Andrew Sansom authored Sep 18, 2025


Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: Andrew Sansom <qthequartermasterman@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

9a4600e4

17 Sep, 2025 1 commit

[Bugfix] Refactor Flashinfer TRTLLM attention kernel selection logic (#24600) · e67a79db

elvischenv authored Sep 18, 2025


Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

e67a79db

16 Sep, 2025 1 commit
- [Deprecation] Remove DeepGEMM Old Symbol Wrapper (#24902) · de2cc3d8
  Wentao Ye authored Sep 15, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  de2cc3d8
13 Sep, 2025 1 commit
- [Core][Multimodal] Cache `supports_kw` (#24773) · 1da0f144
  Lukas Geiger authored Sep 13, 2025
```
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
```
  1da0f144
12 Sep, 2025 1 commit
- [Core] Shared memory based object store for Multimodal data caching and IPC (#20452) · a5b84f1c
  dongluw authored Sep 12, 2025
```
Signed-off-by: donglu <donglu@cohere.com>
```
  a5b84f1c
11 Sep, 2025 2 commits

[flashinfer] [kernel] support for fp8 kv cache for trtllm prefill attention (#24197) · e42af78b
Xiaozhu Meng authored Sep 11, 2025
```
Signed-off-by: Xiaozhu <mxz297@gmail.com>
```
e42af78b

Allow users to specify kv cache memory size (#21489) · 94e6b2d5

Boyuan Feng authored Sep 11, 2025

Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

94e6b2d5

09 Sep, 2025 3 commits
- [ROCm][Feature] Enable Pipeline Parallelism with Ray Compiled Graph on ROCm (#24275) · 73e688cb
  Charlie Fu authored Sep 09, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
```
  73e688cb
- [Misc] Improve Worker process title and logging prefix (#22205) · 0cdd2136
  22quinn authored Sep 08, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  0cdd2136
- [Core] Use sha256 bytes instead of BlockHash to reduce GC overhead (#23673) · 82dfb12e
  Zebing Lin authored Sep 09, 2025
```
Signed-off-by: linzebing <linzebing1995@gmail.com>
```
  82dfb12e
31 Aug, 2025 1 commit
- [Minor] Fix some random typos in comments (#24009) · 752d2e1c
  Nick Hill authored Aug 31, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  752d2e1c
30 Aug, 2025 2 commits

Fix wrong truncate_prompt_tokens type hint (#22761) · 5b8077b8

Gabriel Marinho authored Aug 30, 2025

Signed-off-by: Gabriel Marinho <gmarinho@ibm.com>
Signed-off-by: Gabriel Marinho <104592062+gmarinho2@users.noreply.github.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>

5b8077b8

[Bugfix] Fix --config arg expansion called from api_server.py (#23944) · 5b31cb17

dubejf authored Aug 30, 2025


Signed-off-by: Jean-Francois Dube <dubejf+gh@gmail.com>
Co-authored-by: Jean-Francois Dube <dubejf+gh@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

5b31cb17

29 Aug, 2025 1 commit
- Adds `json_count_leaves` utility function (#23899) · 4f7cde72
  Adit Chawdhary authored Aug 29, 2025
```
Signed-off-by: aditchawdhary <aditxy@hotmail.com>
```
  4f7cde72
28 Aug, 2025 1 commit
- [Log] Use Debug Once for DeepGEMM E8M0 When not Enabled (#23858) · d3d2aad5
  Wentao Ye authored Aug 28, 2025
  
  d3d2aad5
27 Aug, 2025 2 commits
- [Feature] Add Hopper DeepGEMM E8M0 for DeepSeekV3.1 scale_fmt (#23666) · 3af47c3c
  Wentao Ye authored Aug 27, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  3af47c3c
- [Bugfix] fix when config.yaml config value is list parse error (#23528) · 8dbf6ed7
  rongfu.leng authored Aug 27, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  8dbf6ed7
26 Aug, 2025 2 commits
- [Kernel] Added flashinfer fp8 per-tensor gemms (#22895) · f66673a3
  nvjullin authored Aug 26, 2025
```
Signed-off-by: Julien Lin <jullin@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  f66673a3
- [Bug] Fix DeepGEMM Env Control (#23591) · 56dcf4e7
  Wentao Ye authored Aug 25, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  56dcf4e7
22 Aug, 2025 4 commits
- add an env var for path to pre-downloaded flashinfer cubin files (#22675) · da65bec3
  Shiyan Deng authored Aug 22, 2025
  
  da65bec3
- [Doc]: fix various typos in multiple files (#23179) · 22cf679a
  Didier Durand authored Aug 22, 2025
```
Signed-off-by: Didier Durand <durand.didier@gmail.com>
```
  22cf679a
- [Bugfix] Fix pooling models on CPU backend (#23392) · 88016c37
  Li, Jiang authored Aug 22, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  88016c37
- [Feature] Enable DeepGEMM Linear on B200; 1.5% E2E throughput improvement (#23351) · 394591e3
  Wentao Ye authored Aug 22, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  394591e3
21 Aug, 2025 1 commit
- [Bugfix] Fix port conflict by obtaining a list of open ports upfront (#21894) · 10f535c0
  Ming Yang authored Aug 21, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
```
  10f535c0
19 Aug, 2025 1 commit

[NVIDIA] Support Flashinfer TRTLLM FP8-q/kv/out Attention Kernel (#21716) · 03752dba

elvischenv authored Aug 19, 2025

Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

03752dba

16 Aug, 2025 2 commits

[V1] Logits processors extensibility (#19912) · bf7f470b

afeldman-nm authored Aug 16, 2025


Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Signed-off-by: Andrew Feldman <afeld2012@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

bf7f470b

[Kernel] Simplify `get_kv_cache_layout` and cache `use_trtllm_attention` env-dependent bit (#22735) · 070da660
Nicolò Lucchesi authored Aug 16, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
070da660