Commits · 047797ef904fd77b13af3a972e92ef6de37a36db · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "2e6705784f3feb49e1100cc2022ff7ca7f899020"

23 Apr, 2025 9 commits

[Bugfix] Triton FA function takes no keyword arguments (#16902) · 047797ef
vllmellm authored Apr 23, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
047797ef

[INTEL-HPU][v0] Port delayed sampling to upstream (#16949) · 56a73526

Chendi.Xue authored Apr 22, 2025


Signed-off-by: Michal Adamczyk <michal.adamczyk@intel.com>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>

56a73526

[misc] tune some env vars for GB200 (#16992) · e1cf90e0
youkaichao authored Apr 23, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
e1cf90e0
Revert "[Misc] Add S3 environment variables for better support of MinIO." (#17021) · 6bc1e30e
Chauncey authored Apr 23, 2025

6bc1e30e
[BugFix] Revert ROCm Custom Paged Attention Env Flag Check (#17022) · 7e081ba7
vllmellm authored Apr 23, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
7e081ba7
[V1][DP] More robust DP/EP dummy request coordination (#16277) · 1e013fa3
Nick Hill authored Apr 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
1e013fa3

[Kernel][ROCM] Upstream prefix prefill speed up for vLLM V1 (#13305) · bc7c4d20

Aleksandr Malyshev authored Apr 22, 2025


Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: root <root@banff-cyxtera-s73-5.ctr.dcgpu>
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: root <root@banff-cyxtera-s65-4.amd.com>
Signed-off-by: maleksan85 <maleksan@amd.com>
Signed-off-by: <>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: root <root@banff-cyxtera-s73-5.ctr.dcgpu>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
Co-authored-by: root <root@banff-cyxtera-s65-4.amd.com>

bc7c4d20

[Bugfix] validate urls object for multimodal content parts (#16990) · 36fe7876
Guillaume Calmettes authored Apr 23, 2025
```
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
36fe7876
[Core][V1][TPU] Enable structured decoding on TPU V1 (#16499) · 83d93371
Chenyaaang authored Apr 22, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
83d93371

22 Apr, 2025 28 commits
- [BugFix] Remove default multiproc executor `collective_rpc` timeout (#17000) · 5175b884
  Nick Hill authored Apr 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  5175b884
- Add assertion for no objects while hashing hf_config (#16930) · 7f58fb97
  Richard Zou authored Apr 22, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  7f58fb97
- [FEAT][ROCm]: Support AITER MLA (#15893) · 30bc3e0f
  vllmellm authored Apr 23, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
```
  30bc3e0f
- [frontend] enhance tool_calls type check (#16882) · f3441071
  Reid authored Apr 22, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  f3441071
- [Misc] Add S3 environment variables for better support of MinIO. (#16977) · 68d4c332
  Chauncey authored Apr 22, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  68d4c332
- [BugFix] Pass in correct VLLM config in FlashInfer backend (#13207) (#16973) · f961d7f6
  Zhengyuan Su (苏政渊) authored Apr 22, 2025
```
Signed-off-by: 苏政渊 <suzhengyuan@moonshot.cn>
Co-authored-by: 苏政渊 <suzhengyuan@moonshot.cn>
```
  f961d7f6
- Improve configs - `SpeculativeConfig` (#16971) · d0591104
  Harry Mellor authored Apr 22, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d0591104
- [Bugfix] Fix distributed bug again in Qwen2.5-VL & Qwen2.5-Omni (#16974) · 571e8dd6
  Yang Fan authored Apr 22, 2025
```
Signed-off-by: fyabc <suyang.fy@alibaba-inc.com>
```
  571e8dd6
- [FEAT][ROCm] Integrate Paged Attention Kernel from AITER (#15001) · 0e237f00
  vllmellm authored Apr 22, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  0e237f00
- [Doc] Improve documentation for multimodal CLI args (#16960) · 8f7bace7
  Cyrus Leung authored Apr 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f7bace7
- [BugFix] Fix incremental detokenization perf issue (#16963) · e4d61442
  Nick Hill authored Apr 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  e4d61442
- [Kernel] Support Microsoft Runtime Kernel Lib for our Low Precision Computation - BitBLAS (#6036) · 8d32dc60
  Lei Wang authored Apr 22, 2025
```
Signed-off-by: xinyuxiao <xinyuxiao2024@gmail.com>
Co-authored-by: xinyuxiao <xinyuxiao2024@gmail.com>
```
  8d32dc60
- [V1] Remove pre-allocation for KV cache (#16941) · c4ab9f3e
  Woosuk Kwon authored Apr 22, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  c4ab9f3e
- [Model] Use autoweightloader for mamba (#16950) · 2689d5c0
  Flora Feng authored Apr 22, 2025
```
Signed-off-by: sfeng33 <4florafeng@gmail.com>
```
  2689d5c0
- [Bugfix] Fix the issue where llm.generate cannot be called repeatedly after... · acba33a0
  Chauncey authored Apr 22, 2025
```
[Bugfix] Fix the issue where llm.generate cannot be called repeatedly after setting GuidedDecodingParams (#16767)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
```
  acba33a0
- [Perf] Optimize `_update_states` for GPU model runner (#16910) · a114bf20
  SnowCharm authored Apr 22, 2025
```
Signed-off-by: snowcharm <snowcharmqq@gmail.com>
```
  a114bf20
- [Bugfix] Fix f-string for Python 3.9-3.11 (#16962) · d6da9322
  Cyrus Leung authored Apr 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d6da9322
- Support S3 Sharded loading with RunAI Model Streamer (#16317) · 71ce4404
  omer-dayan authored Apr 22, 2025
```
Signed-off-by: Omer Dayan (SW-GPU) <omer@run.ai>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  71ce4404
- [Performance][ROCm] Add skinny gemms for unquantized linear on ROCm (#15830) · 188b7f9b
  Charlie Fu authored Apr 21, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
```
  188b7f9b
- [V1] Remove additional_config check (#16710) · b9b47469
  wangxiyuan authored Apr 22, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  b9b47469
- [Kernel] Add expert_map support to Cutlass FP8 MOE (#16861) · 7b8a2ab7
  Varun Sundar Rabindranath authored Apr 21, 2025
```
Signed-off-by: varun sundar rabindranath <vsundarr@redhat.com>
Co-authored-by: varun sundar rabindranath <vsundarr@redhat.com>
```
  7b8a2ab7
- [Misc] Remove the chunked prefill warning for LoRA (#16925) · c9acbf11
  Jee Jee Li authored Apr 22, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  c9acbf11
- [ROCm] Add aiter tkw1 kernel for Llama4 fp8 (#16727) · 5b794cae
  kliuae authored Apr 22, 2025
```
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  5b794cae
- [Bugfix]: fix issue with n>1 sampling on v1 requests overriding each other (#16863) · 0e425449
  Jeffrey Li authored Apr 21, 2025
```
Signed-off-by: Jeffrey Li <jeffrey.dot.li@gmail.com>
```
  0e425449
- [BugFix][Spec Decode] No in-place update to draft probs (#16952) · 1311913f
  Woosuk Kwon authored Apr 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  1311913f
- [TPU][V1] Enable Top-P (#16843) · fa3bba2a
  Nicolò Lucchesi authored Apr 22, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  fa3bba2a
- [V1] V1 FlashInfer Attention (#16684) · 986537f1
  Michael Goin authored Apr 21, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Aurick Qiao <qiao@aurick.net>
```
  986537f1
- [TPU][V1] Capture multimodal encoder during model compilation (#15051) · 21020752
  Nicolò Lucchesi authored Apr 22, 2025
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Siyuan Liu <lsiyuan@google.com>
```
  21020752
21 Apr, 2025 3 commits
- [TPU][V1] Implicitly adjust page size when there's SMEM OOM (#16871) · 471fe656
  Chengji Yao authored Apr 21, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  471fe656
- [V1][Spec Decode] Handle draft tokens beyond max_model_len (#16087) · 3a0fba5c
  Woosuk Kwon authored Apr 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3a0fba5c
- [Core] Speed up decode by remove synchronizing operation in sampler (#16436) · 299ebb62
  Chanh Nguyen authored Apr 21, 2025
```
Signed-off-by: Chanh Nguyen <cnguyen@linkedin.com>
Co-authored-by: Chanh Nguyen <cnguyen@linkedin.com>
```
  299ebb62