Commits · 0c0fdae84f1da5e45518aafc7b32e8139055adae · OpenDAS / vllm_cscc · GitLab

09 May, 2025 17 commits
- [Hardware/NVIDIA/Kernel] Enable nvidia/DeepSeek-R1-FP4 Model (#16362) · 0c0fdae8
  Pavani Majety authored May 09, 2025
  
  0c0fdae8
- Improve configs - the rest! (#17562) · 4b2ed792
  Harry Mellor authored May 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  4b2ed792
- [V1][Spec Decoding] Include bonus tokens in mean acceptance length (#17908) · 7e357113
  Mark McLoughlin authored May 09, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  7e357113
- Add option to use torch._inductor.standalone_compile (#17057) · ea2236bf
  Richard Zou authored May 09, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  ea2236bf
- Handle error when `str` passed to `/v1/audio/transcriptions` (#17909) · 7d4aedae
  Harry Mellor authored May 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  7d4aedae
- Update CT WNA16MarlinMoE integration (#16666) · 22481fbf
  Michael Goin authored May 09, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  22481fbf
- [Misc] Auto fallback to float16 for pre-Ampere GPUs when detected bfloat16 config (#17265) · 5c4c08f6
  Isotr0py authored May 10, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  5c4c08f6
- Revert "[BugFix][AMD] Compatible patch for latest AITER(05/07/2025)" (#17910) · 85b72cb7
  Michael Goin authored May 09, 2025
  
  85b72cb7
- [v1] Move block management logic from KVCacheManager to SpecializedManager (#17474) · 200da9a5
  Chen Zhang authored May 09, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  200da9a5
- [BugFix][AMD] Compatible patch for latest AITER(05/07/2025) (#17864) · 9f64e934
  qli88 authored May 09, 2025
```
Signed-off-by: Qiang Li <qiang.li2@amd.com>
```
  9f64e934
- Change `top_k` to be disabled with `0` (still accept `-1` for now) (#17773) · c6798baa
  Harry Mellor authored May 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  c6798baa
- Fix Whisper crash caused by invalid``` max_num_batched_tokens``` config (#17853) · 5b2dcbf0
  inkcherry authored May 09, 2025
```
Signed-off-by: inkcherry <mingzhi.liu@intel.com>
```
  5b2dcbf0
- [Bugfix][CPU] Fix broken AVX2 CPU TP support (#17252) · 6e4a93e3
  Isotr0py authored May 09, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  6e4a93e3
- [Bugfix][ROCm] Fix AITER MLA V1 (#17880) · 217db4ba
  vllmellm authored May 09, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  217db4ba
- [BUGFIX]: return fast when request requires prompt logprobs (#17251) · d310e6de
  Ning Xie authored May 09, 2025
  
  d310e6de
- [Attention] MLA move rotary embedding to cuda-graph region (#17668) · 5e6f9394
  Lucas Wilkinson authored May 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  5e6f9394
- [FEAT][ROCm]: Support AITER MLA on V1 Engine (#17523) · 3c9396a6
  vllmellm authored May 09, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
```
  3c9396a6
08 May, 2025 18 commits
- Add cutlass support for blackwell fp8 blockwise gemm (#14383) · 376786fa
  Shu Wang authored May 08, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
```
  376786fa
- Fix noisy warning for uncalibrated q_scale/p_scale (#17414) · 4f605a6d
  Michael Goin authored May 08, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  4f605a6d
- [V1] Improve VLLM_ALLOW_INSECURE_SERIALIZATION logging (#17860) · 226a4272
  Russell Bryant authored May 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  226a4272
- [Misc] Delete LoRA-related redundancy code (#17841) · a944f8ed
  Jee Jee Li authored May 08, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a944f8ed
- [Bugfix] `use_fast` failing to be propagated to Qwen2-VL image processor (#17838) · 015815fe
  Cyrus Leung authored May 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  015815fe
- [Bugfix] Fix quark fp8 format loading on AMD GPUs (#12612) · bb239a73
  fxmarty-amd authored May 08, 2025
```
Signed-off-by: Felix Marty <felmarty@amd.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
```
  bb239a73
- [TPU] Fix the test_sampler (#17820) · a463555d
  Jevin Jiang authored May 08, 2025
  
  a463555d
- [Misc] support model prefix & add deepseek vl2 tiny fused moe config (#17763) · 0a9bbaa1
  xsank authored May 08, 2025
```
Signed-off-by: 唯勤 <xsank.mz@alibaba-inc.com>
Co-authored-by: 唯勤 <xsank.mz@alibaba-inc.com>
```
  0a9bbaa1
- [Bugfix] Fix bad words for Mistral models (#17753) · 39956efb
  Qiong Zhou Huang authored May 07, 2025
```
Signed-off-by: Qiong Zhou Huang <qiong@phonic.co>
```
  39956efb
- [Qwen3]add qwen3-235b-bf16 fused moe config on A100 (#17715) · 597051e5
  Ximingwang-09 authored May 08, 2025
  
  597051e5
- [Frontend] Chat template fallbacks for multimodal models (#17805) · 96722aa8
  Cyrus Leung authored May 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  96722aa8
- [Hardware][Intel-Gaudi] Support Automatic Prefix Caching on HPU (#17648) · 843b2227
  Agata Dobrzyniewicz authored May 08, 2025
```
Signed-off-by: Agata Dobrzyniewicz <adobrzyniewicz@habana.ai>
```
  843b2227
- [Kernel][Hardware][AMD] Bf16 mfma opt for ROCm skinny GEMMs (#17071) · 5a499e70
  Hashem Hashemi authored May 07, 2025
```
Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: charlifu <charlifu@amd.com>
```
  5a499e70
- [V1] Add VLLM_ALLOW_INSECURE_SERIALIZATION env var (#17490) · 6930a411
  Russell Bryant authored May 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  6930a411
- Only log non-default CLI args for online serving (#17803) · 998eea4a
  Harry Mellor authored May 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  998eea4a
- Improve exception reporting in MP engine (#17800) · b2da14a0
  Vadim Markovtsev authored May 08, 2025
```
Signed-off-by: Vadim Markovtsev <vadim@poolside.ai>
```
  b2da14a0
- [Core] Support full cuda graph in v1 (#16072) · 7ea2adb8
  Chanh Nguyen authored May 07, 2025
```
Signed-off-by: Chanh Nguyen <cnguyen@linkedin.com>
Co-authored-by: Chanh Nguyen <cnguyen@linkedin.com>
```
  7ea2adb8
- [BugFix] Fix `--disable-log-stats` in V1 server mode (#17600) · 3d13ca0e
  Nick Hill authored May 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  3d13ca0e
07 May, 2025 5 commits
- [Core][Feature] Input metadata dump on crash (#13407) · d43f914d
  Wallas Henrique authored May 07, 2025
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  d43f914d
- [BugFix] Avoid secondary missing `MultiprocExecutor.workers` error (#17811) · ed5272cf
  Nick Hill authored May 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  ed5272cf
- [Hardware][TPU][V1] Multi-LoRA implementation for the V1 TPU backend (#14238) · c20ef40f
  Akshat Tripathi authored May 07, 2025
```
Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
```
  c20ef40f
- [Quantization] Quark MXFP4 format loading (#16943) · db593aa6
  Bowen Bao authored May 07, 2025
  
  db593aa6
- [Bugfix] Fix missing lora name mapping for lora without prefix (#17793) · f98e3075
  Isotr0py authored May 08, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  f98e3075