Commits · c44c384b1c85b2e83c7b1998fb2a3c1c981fdf6b · OpenDAS / vllm_cscc · GitLab

09 May, 2025 10 commits
- Revert "[BugFix][AMD] Compatible patch for latest AITER(05/07/2025)" (#17910) · 85b72cb7
  Michael Goin authored May 09, 2025
  
  85b72cb7
- [v1] Move block management logic from KVCacheManager to SpecializedManager (#17474) · 200da9a5
  Chen Zhang authored May 09, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  200da9a5
- [BugFix][AMD] Compatible patch for latest AITER(05/07/2025) (#17864) · 9f64e934
  qli88 authored May 09, 2025
```
Signed-off-by: Qiang Li <qiang.li2@amd.com>
```
  9f64e934
- Change `top_k` to be disabled with `0` (still accept `-1` for now) (#17773) · c6798baa
  Harry Mellor authored May 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  c6798baa
- Fix Whisper crash caused by invalid``` max_num_batched_tokens``` config (#17853) · 5b2dcbf0
  inkcherry authored May 09, 2025
```
Signed-off-by: inkcherry <mingzhi.liu@intel.com>
```
  5b2dcbf0
- [Bugfix][CPU] Fix broken AVX2 CPU TP support (#17252) · 6e4a93e3
  Isotr0py authored May 09, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  6e4a93e3
- [Bugfix][ROCm] Fix AITER MLA V1 (#17880) · 217db4ba
  vllmellm authored May 09, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  217db4ba
- [BUGFIX]: return fast when request requires prompt logprobs (#17251) · d310e6de
  Ning Xie authored May 09, 2025
  
  d310e6de
- [Attention] MLA move rotary embedding to cuda-graph region (#17668) · 5e6f9394
  Lucas Wilkinson authored May 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  5e6f9394
- [FEAT][ROCm]: Support AITER MLA on V1 Engine (#17523) · 3c9396a6
  vllmellm authored May 09, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
```
  3c9396a6
08 May, 2025 18 commits
- Add cutlass support for blackwell fp8 blockwise gemm (#14383) · 376786fa
  Shu Wang authored May 08, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
```
  376786fa
- Fix noisy warning for uncalibrated q_scale/p_scale (#17414) · 4f605a6d
  Michael Goin authored May 08, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  4f605a6d
- [V1] Improve VLLM_ALLOW_INSECURE_SERIALIZATION logging (#17860) · 226a4272
  Russell Bryant authored May 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  226a4272
- [Misc] Delete LoRA-related redundancy code (#17841) · a944f8ed
  Jee Jee Li authored May 08, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a944f8ed
- [Bugfix] `use_fast` failing to be propagated to Qwen2-VL image processor (#17838) · 015815fe
  Cyrus Leung authored May 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  015815fe
- [Bugfix] Fix quark fp8 format loading on AMD GPUs (#12612) · bb239a73
  fxmarty-amd authored May 08, 2025
```
Signed-off-by: Felix Marty <felmarty@amd.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
```
  bb239a73
- [TPU] Fix the test_sampler (#17820) · a463555d
  Jevin Jiang authored May 08, 2025
  
  a463555d
- [Misc] support model prefix & add deepseek vl2 tiny fused moe config (#17763) · 0a9bbaa1
  xsank authored May 08, 2025
```
Signed-off-by: 唯勤 <xsank.mz@alibaba-inc.com>
Co-authored-by: 唯勤 <xsank.mz@alibaba-inc.com>
```
  0a9bbaa1
- [Bugfix] Fix bad words for Mistral models (#17753) · 39956efb
  Qiong Zhou Huang authored May 07, 2025
```
Signed-off-by: Qiong Zhou Huang <qiong@phonic.co>
```
  39956efb
- [Qwen3]add qwen3-235b-bf16 fused moe config on A100 (#17715) · 597051e5
  Ximingwang-09 authored May 08, 2025
  
  597051e5
- [Frontend] Chat template fallbacks for multimodal models (#17805) · 96722aa8
  Cyrus Leung authored May 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  96722aa8
- [Hardware][Intel-Gaudi] Support Automatic Prefix Caching on HPU (#17648) · 843b2227
  Agata Dobrzyniewicz authored May 08, 2025
```
Signed-off-by: Agata Dobrzyniewicz <adobrzyniewicz@habana.ai>
```
  843b2227
- [Kernel][Hardware][AMD] Bf16 mfma opt for ROCm skinny GEMMs (#17071) · 5a499e70
  Hashem Hashemi authored May 07, 2025
```
Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: charlifu <charlifu@amd.com>
```
  5a499e70
- [V1] Add VLLM_ALLOW_INSECURE_SERIALIZATION env var (#17490) · 6930a411
  Russell Bryant authored May 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  6930a411
- Only log non-default CLI args for online serving (#17803) · 998eea4a
  Harry Mellor authored May 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  998eea4a
- Improve exception reporting in MP engine (#17800) · b2da14a0
  Vadim Markovtsev authored May 08, 2025
```
Signed-off-by: Vadim Markovtsev <vadim@poolside.ai>
```
  b2da14a0
- [Core] Support full cuda graph in v1 (#16072) · 7ea2adb8
  Chanh Nguyen authored May 07, 2025
```
Signed-off-by: Chanh Nguyen <cnguyen@linkedin.com>
Co-authored-by: Chanh Nguyen <cnguyen@linkedin.com>
```
  7ea2adb8
- [BugFix] Fix `--disable-log-stats` in V1 server mode (#17600) · 3d13ca0e
  Nick Hill authored May 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  3d13ca0e
07 May, 2025 12 commits
- [Core][Feature] Input metadata dump on crash (#13407) · d43f914d
  Wallas Henrique authored May 07, 2025
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  d43f914d
- [BugFix] Avoid secondary missing `MultiprocExecutor.workers` error (#17811) · ed5272cf
  Nick Hill authored May 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  ed5272cf
- [Hardware][TPU][V1] Multi-LoRA implementation for the V1 TPU backend (#14238) · c20ef40f
  Akshat Tripathi authored May 07, 2025
```
Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
```
  c20ef40f
- [Quantization] Quark MXFP4 format loading (#16943) · db593aa6
  Bowen Bao authored May 07, 2025
  
  db593aa6
- [Bugfix] Fix missing lora name mapping for lora without prefix (#17793) · f98e3075
  Isotr0py authored May 08, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  f98e3075
- Fix and simplify `deprecated=True` CLI `kwarg` (#17781) · 646a31e5
  Harry Mellor authored May 07, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  646a31e5
- [Bugfix] Fix Video IO error for short video (#17791) · be8ff88e
  Isotr0py authored May 07, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  be8ff88e
- [ROCm][FP8][Kernel] FP8 quantization fused into Custom Paged Attention (#17139) · 32aa74c0
  Gregory Shtrasberg authored May 07, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  32aa74c0
- Make key optional for rotary embedding (#17566) · 98c89e16
  Yong Hoon Shin authored May 07, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  98c89e16
- Add NeuronxDistributedInference support, Speculative Decoding, Dynamic on-device sampling (#16357) · 043e4c49
  Satyajith Chilappagari authored May 07, 2025
```
Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
Co-authored-by: Aaron Dou <yzdou@amazon.com>
Co-authored-by: Shashwat Srijan <sssrijan@amazon.com>
Co-authored-by: Chongming Ni <chongmni@amazon.com>
Co-authored-by: Amulya Ballakur <amulyaab@amazon.com>
Co-authored-by: Patrick Lange <patlange@amazon.com>
Co-authored-by: Elaine Zhao <elaineyz@amazon.com>
Co-authored-by: Lin Lin Pan <tailinpa@amazon.com>
Co-authored-by: Navyadhara Gogineni <navyadha@amazon.com>
Co-authored-by: Yishan McNabb <yishanm@amazon.com>
Co-authored-by: Mrinal Shukla <181322398+mrinalks@users.noreply.github.com>
```
  043e4c49
- [Misc] Remove qlora_adapter_name_or_path (#17699) · ba7703e6
  Jee Jee Li authored May 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  ba7703e6
- [Kernel] Use fused rmsnorm for some models like qwen3 series (#17735) · f80ae5bd
  Wanrui Dai authored May 07, 2025
```
Signed-off-by: evian <eviantai@u.nus.edu>
Co-authored-by: evian <eviantai@u.nus.edu>
```
  f80ae5bd