Commits · 94e6b2d55f7fd539f01591c7521ec104b25f4e68 · OpenDAS / vllm_cscc

11 Sep, 2025 1 commit

Allow users to specify kv cache memory size (#21489) · 94e6b2d5

Boyuan Feng authored Sep 11, 2025

Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

94e6b2d5

09 Sep, 2025 3 commits
- [ROCm][Feature] Enable Pipeline Parallelism with Ray Compiled Graph on ROCm (#24275) · 73e688cb
  Charlie Fu authored Sep 09, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
```
  73e688cb
- [Misc] Improve Worker process title and logging prefix (#22205) · 0cdd2136
  22quinn authored Sep 08, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  0cdd2136
- [Core] Use sha256 bytes instead of BlockHash to reduce GC overhead (#23673) · 82dfb12e
  Zebing Lin authored Sep 09, 2025
```
Signed-off-by: linzebing <linzebing1995@gmail.com>
```
  82dfb12e
31 Aug, 2025 1 commit
- [Minor] Fix some random typos in comments (#24009) · 752d2e1c
  Nick Hill authored Aug 31, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  752d2e1c
30 Aug, 2025 2 commits

Fix wrong truncate_prompt_tokens type hint (#22761) · 5b8077b8

Gabriel Marinho authored Aug 30, 2025

Signed-off-by: Gabriel Marinho <gmarinho@ibm.com>
Signed-off-by: Gabriel Marinho <104592062+gmarinho2@users.noreply.github.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>

5b8077b8

[Bugfix] Fix --config arg expansion called from api_server.py (#23944) · 5b31cb17

dubejf authored Aug 30, 2025


Signed-off-by: Jean-Francois Dube <dubejf+gh@gmail.com>
Co-authored-by: Jean-Francois Dube <dubejf+gh@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

5b31cb17

29 Aug, 2025 1 commit
- Adds `json_count_leaves` utility function (#23899) · 4f7cde72
  Adit Chawdhary authored Aug 29, 2025
```
Signed-off-by: aditchawdhary <aditxy@hotmail.com>
```
  4f7cde72
28 Aug, 2025 1 commit
- [Log] Use Debug Once for DeepGEMM E8M0 When not Enabled (#23858) · d3d2aad5
  Wentao Ye authored Aug 28, 2025
  
  d3d2aad5
27 Aug, 2025 2 commits
- [Feature] Add Hopper DeepGEMM E8M0 for DeepSeekV3.1 scale_fmt (#23666) · 3af47c3c
  Wentao Ye authored Aug 27, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  3af47c3c
- [Bugfix] fix when config.yaml config value is list parse error (#23528) · 8dbf6ed7
  rongfu.leng authored Aug 27, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  8dbf6ed7
26 Aug, 2025 2 commits
- [Kernel] Added flashinfer fp8 per-tensor gemms (#22895) · f66673a3
  nvjullin authored Aug 26, 2025
```
Signed-off-by: Julien Lin <jullin@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  f66673a3
- [Bug] Fix DeepGEMM Env Control (#23591) · 56dcf4e7
  Wentao Ye authored Aug 25, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  56dcf4e7
22 Aug, 2025 4 commits
- add an env var for path to pre-downloaded flashinfer cubin files (#22675) · da65bec3
  Shiyan Deng authored Aug 22, 2025
  
  da65bec3
- [Doc]: fix various typos in multiple files (#23179) · 22cf679a
  Didier Durand authored Aug 22, 2025
```
Signed-off-by: Didier Durand <durand.didier@gmail.com>
```
  22cf679a
- [Bugfix] Fix pooling models on CPU backend (#23392) · 88016c37
  Li, Jiang authored Aug 22, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  88016c37
- [Feature] Enable DeepGEMM Linear on B200; 1.5% E2E throughput improvement (#23351) · 394591e3
  Wentao Ye authored Aug 22, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  394591e3
21 Aug, 2025 1 commit
- [Bugfix] Fix port conflict by obtaining a list of open ports upfront (#21894) · 10f535c0
  Ming Yang authored Aug 21, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
```
  10f535c0
19 Aug, 2025 1 commit

[NVIDIA] Support Flashinfer TRTLLM FP8-q/kv/out Attention Kernel (#21716) · 03752dba

elvischenv authored Aug 19, 2025

Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

03752dba

16 Aug, 2025 3 commits

[V1] Logits processors extensibility (#19912) · bf7f470b

afeldman-nm authored Aug 16, 2025


Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Signed-off-by: Andrew Feldman <afeld2012@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

bf7f470b

[Kernel] Simplify `get_kv_cache_layout` and cache `use_trtllm_attention` env-dependent bit (#22735) · 070da660
Nicolò Lucchesi authored Aug 16, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
070da660
[Misc] Support passing multiple request ids at once to `AsyncLLM.abort()` (#22944) · ad0297d1
Nick Hill authored Aug 15, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
ad0297d1

15 Aug, 2025 3 commits

[BugFix] Make `run_once` thread-safe (#22978) · 236b864e

Yichen Yan authored Aug 16, 2025



Signed-off-by: <wenji.yyc@alibaba-inc.com>
Signed-off-by: Yichen Yan <wenji.yyc@alibaba-inc.com>

236b864e

[v1] Move block_hashes from KVCacheManager to Request.block_hashes (#19728) · c280066f
Or Ozeri authored Aug 16, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
c280066f

[V1] [Hybrid] Support using float32 for state in Hybrid Models (Mamba2, Mamba1, Minimax) (#22928) · 75531a6c

Thomas Parnell authored Aug 15, 2025


Signed-off-by: Daniel Afrimi <danielafrimi8@gmail.com>
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Daniel Afrimi <danielafrimi8@gmail.com>
Co-authored-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>

75531a6c

14 Aug, 2025 2 commits

[Kernel] Add nvfp4 gemm flashinfer backends (#22346) · 279a5f31

nvjullin authored Aug 15, 2025


Signed-off-by: Julien Lin <jullin@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>

279a5f31

[BugFix] Threadsafe close async zmq sockets (#22877) · eb08487b

Nick Hill authored Aug 14, 2025


Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

eb08487b

13 Aug, 2025 2 commits

[Bugfix] Fix MiniCPMV Image input inference failed (#22813) · 9bd9294f

HWH authored Aug 14, 2025


Signed-off-by: HWH <67449739+jio-H@users.noreply.github.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

9bd9294f

Force TRTLLM attention for gpt-oss on SM100 (#22678) · c6b92879
Michael Goin authored Aug 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
c6b92879

11 Aug, 2025 3 commits
- [Feature] Add `VLLM_USE_DEEP_GEMM_E8M0` Env to Control E8M0 Scale (#21968) · f7dcce7a
  Wentao Ye authored Aug 11, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  f7dcce7a
- [Misc] Move jsontree to utils (#22622) · 951b0382
  Cyrus Leung authored Aug 11, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  951b0382
- [Docs] Add comprehensive CLI reference for all large `vllm` subcommands (#22601) · bc1d02ac
  Harry Mellor authored Aug 11, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  bc1d02ac
10 Aug, 2025 1 commit

Migrate LlavaNextImageInputs to TensorSchema (#21774) · b4e29167

Benji Beck authored Aug 10, 2025


Signed-off-by: Benji Beck <benjibeck@meta.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

b4e29167

09 Aug, 2025 2 commits
- [Docs] Reduce noise in docs and `--help` from the JSON tip (#22567) · 56186474
  Harry Mellor authored Aug 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  56186474
- [Log] Add Warning for Deprecation of DeepGEMM old version (#22194) · 3157aebb
  Wentao Ye authored Aug 09, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  3157aebb
08 Aug, 2025 4 commits

[gpt-oss] triton kernel mxfp4 (#22421) · e789cad6

Yongye Zhu authored Aug 08, 2025



Signed-off-by: <zyy1102000@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>

e789cad6

Remove exception for Python 3.8 typing from linter (#22506) · e5ebeeba
Harry Mellor authored Aug 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
e5ebeeba
[BugFix] Don't cancel asyncio tasks directly from destructors (#22476) · ccdae737
Nick Hill authored Aug 08, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
ccdae737

Support Tensorrt-LLM MoE fp4 for low-latency (#21331) · a3b9c17b

Shu Wang authored Aug 07, 2025


Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: XIn Li <xinli@nvidia.com>
Co-authored-by: XIn Li <xinli@nvidia.com>

a3b9c17b

07 Aug, 2025 1 commit

Fix trtllm-gen attention env and add attention sink (#22378) · 9a3835aa

Lain authored Aug 06, 2025


Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
Signed-off-by: Lain <fusiyuan2000@hotmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>

9a3835aa