Commits · 7e8977fcd4e9c3bf6b114c7dc715b28a61b5cdb0 · OpenDAS / vllm_cscc

18 Jun, 2025 1 commit

[Hardware][AMD] integrate aiter chunked prefill into vllm (#18596) · 8b6e1d63

Zzz9990 authored Jun 18, 2025


Signed-off-by: fsx950223 <fsx950223@outlook.com>
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: fsx950223 <fsx950223@outlook.com>
Co-authored-by: charlifu <charlifu@amd.com>

8b6e1d63

17 Jun, 2025 1 commit
- [V1][Kernel] Flashinfer HND KV cache layout (#19280) · 4c8f64fa
  Nicolò Lucchesi authored Jun 17, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  4c8f64fa
12 Jun, 2025 2 commits
- [Misc] Turn MOE_DP_CHUNK_SIZE into an env var (#19506) · 9d880f59
  Varun Sundar Rabindranath authored Jun 12, 2025
  
  9d880f59
- [torch.compile][ROCm] Fuse quantization onto attention using a torch.compile pass (#16756) · f98548b9
  Luka Govedič authored Jun 12, 2025
```
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
```
  f98548b9
10 Jun, 2025 1 commit
- Automatically bind CPU OMP Threads of a rank to CPU ids of a NUMA node. (#17930) · 9368cc90
  Louie Tsai authored Jun 09, 2025
```
Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
Co-authored-by: Li, Jiang <bigpyj64@gmail.com>
```
  9368cc90
09 Jun, 2025 2 commits
- [Frontend] Make TIMEOUT_KEEP_ALIVE configurable through env var (#18472) · 31f58be9
  liusiqian-tal authored Jun 10, 2025
```
Signed-off-by: liusiqian <liusiqian@tal.com>
```
  31f58be9
- [Misc] Fixes and Optimizations for DeepEP + DeepGEMM combination. (#19298) · 5cf2daea
  Varun Sundar Rabindranath authored Jun 09, 2025
```
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>
```
  5cf2daea
08 Jun, 2025 1 commit
- [Multi Modal] Add an env var for message queue max chunk bytes (#19242) · cda10fa3
  jennyyyyzhen authored Jun 08, 2025
```
Signed-off-by: yZhen <yZhen@fb.com>
Co-authored-by: yZhen <yZhen@fb.com>
```
  cda10fa3
05 Jun, 2025 1 commit
- [MISC][Bugfix] Use less CPU when message queue has been empty for some time (#16226) · 85e2b7bb
  Povilas Kanapickas authored Jun 05, 2025
```
Signed-off-by: Povilas Kanapickas <povilas@radix.lt>
```
  85e2b7bb
03 Jun, 2025 3 commits
- [Kernel] DeepEP dispatch-combine kernel integration (#18434) · fa98d777
  Varun Sundar Rabindranath authored Jun 03, 2025
```
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
  fa98d777
- [Misc] Add SPDX-FileCopyrightText (#19100) · 02f0c7b2
  Simon Mo authored Jun 03, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  02f0c7b2
- [Hardware][TPU] Initial support of model parallelism with single worker using SPMD (#18011) · 9112b443
  Siyuan Liu authored Jun 02, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Co-authored-by: Hossein Sarshar <hossein.sarshar@gmail.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
```
  9112b443
30 May, 2025 2 commits
- Tool parser regex timeout handling (#18960) · 1dab4d57
  Will Eaton authored May 30, 2025
```
Signed-off-by: Will Eaton <weaton@redhat.com>
```
  1dab4d57
- [Bugfix][Failing Test] Fix test_vllm_port.py (#18618) · 5f1d0c81
  Rabi Mishra authored May 30, 2025
```
Signed-off-by: rabi <ramishra@redhat.com>
```
  5f1d0c81
29 May, 2025 2 commits
- Use standalone_compile by default in torch >= 2.8.0 (#18846) · a521ef06
  Richard Zou authored May 29, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  a521ef06
- [Attention][V1] Toggle for v1 attention backend (#18275) · da4b69d0
  Gregory Shtrasberg authored May 29, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  da4b69d0
24 May, 2025 1 commit
- [Doc] Reorganize user guide (#18661) · 1cb194a0
  Cyrus Leung authored May 24, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1cb194a0
23 May, 2025 3 commits
- [ModelOpt] Introduce VLLM_MAX_TOKENS_PER_EXPERT_FP4_MOE env var to control... · f2036734
  Pavani Majety authored May 23, 2025
```
[ModelOpt] Introduce VLLM_MAX_TOKENS_PER_EXPERT_FP4_MOE env var to control blockscale tensor allocation (#18160)
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  f2036734
- Refactor pplx init logic to make it modular (prepare for deepep) (#18200) · 6a7988c5
  youkaichao authored May 23, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  6a7988c5
- Migrate docs from Sphinx to MkDocs (#18145) · a1fe24d9
  Harry Mellor authored May 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a1fe24d9
16 May, 2025 1 commit
- Throw better error for when running into k8s service discovery issue (#18209) · 6b31c84a
  Will Eaton authored May 16, 2025
```
Signed-off-by: Will Eaton <weaton@redhat.com>
```
  6b31c84a
15 May, 2025 1 commit
- Support custom implementations of VideoLoader backends. (#18091) · 4f07a640
  Chenheli Hua authored May 14, 2025
  
  4f07a640
14 May, 2025 1 commit
- [core][distributed] add ep group and all2all interface (#18077) · 6266c57b
  youkaichao authored May 14, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  6266c57b
12 May, 2025 2 commits

[Lora][Frontend]Add default local directory LoRA resolver plugin. (#16855) · 98ea3560
Jonathan Berkhahn authored May 12, 2025
```
Signed-off-by: jberkhahn <jaberkha@us.ibm.com>
```
98ea3560

[P/D] NIXL Integration (#17751) · d1911020

Robert Shaw authored May 12, 2025


Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Brent Salisbury <bsalisbu@redhat.com>

d1911020

09 May, 2025 1 commit
- Add option to use torch._inductor.standalone_compile (#17057) · ea2236bf
  Richard Zou authored May 09, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  ea2236bf
08 May, 2025 1 commit

[V1] Add VLLM_ALLOW_INSECURE_SERIALIZATION env var (#17490) · 6930a411

Russell Bryant authored May 08, 2025


Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

6930a411

07 May, 2025 1 commit
- [Quantization] Quark MXFP4 format loading (#16943) · db593aa6
  Bowen Bao authored May 07, 2025
  
  db593aa6
24 Apr, 2025 1 commit
- Addendum Fix to support FIPS enabled machines with MD5 hashing (#17043) · ed2e4646
  Areeb Syed authored Apr 24, 2025
```
Signed-off-by: sydarb <areebsyed237@gmail.com>
```
  ed2e4646
23 Apr, 2025 1 commit

[INTEL-HPU][v0] Port delayed sampling to upstream (#16949) · 56a73526

Chendi.Xue authored Apr 22, 2025


Signed-off-by: Michal Adamczyk <michal.adamczyk@intel.com>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>

56a73526

22 Apr, 2025 4 commits

[FEAT][ROCm]: Support AITER MLA (#15893) · 30bc3e0f

vllmellm authored Apr 23, 2025


Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>

30bc3e0f

[FEAT][ROCm] Integrate Paged Attention Kernel from AITER (#15001) · 0e237f00

vllmellm authored Apr 22, 2025


Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>

0e237f00

[Performance][ROCm] Add skinny gemms for unquantized linear on ROCm (#15830) · 188b7f9b
Charlie Fu authored Apr 21, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
```
188b7f9b

[ROCm] Add aiter tkw1 kernel for Llama4 fp8 (#16727) · 5b794cae

kliuae authored Apr 22, 2025


Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>

5b794cae

19 Apr, 2025 1 commit

[Model][VLM] Add Qwen2.5-Omni model support (thinker only) (#15130) · 2c1bd848

Yang Fan authored Apr 19, 2025


Signed-off-by: fyabc <suyang.fy@alibaba-inc.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Xiong Wang <wangxiongts@163.com>

2c1bd848

17 Apr, 2025 2 commits

[V1][TPU] Enable Top K (#15489) · eb5819b2

Nicolò Lucchesi authored Apr 17, 2025


Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
Co-authored-by: Hyesoo Yang <hyeygit@gmail.com>

eb5819b2

[V1][Performance] Implement custom serializaton for MultiModalKwargs [Rebased] (#16432) · 3092375e

Staszek Paśko authored Apr 17, 2025


Signed-off-by: Staszek Pasko <staszek@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

3092375e

09 Apr, 2025 1 commit
- [Core] Upgrade to xgrammar 0.1.18, add cache size limit (#16283) · cb84e45a
  Russell Bryant authored Apr 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  cb84e45a
06 Apr, 2025 1 commit
- [Misc] Improve model redirect to accept json dictionary (#16119) · c2a96715
  Isotr0py authored Apr 06, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  c2a96715
01 Apr, 2025 1 commit
- Add option to use DeepGemm contiguous grouped gemm kernel for fused MoE operations. (#13932) · e59ca942
  bnellnm authored Apr 01, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  e59ca942