Commits · eea22a56ab08fb018e8fc51d1bf988cb85f37115 · OpenDAS / vllm_cscc · GitLab

11 May, 2025 4 commits
- fix amd triton mla path (#17871) · eea22a56
  Shiyan Deng authored May 11, 2025
  
  eea22a56
- [Perf] Use small max_num_batched_tokens for A100 (#17885) · 91121552
  Kuntai Du authored May 11, 2025
```
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
```
  91121552
- [Bugfix] Add revision to `transformers.Auto*.from_pretrained` processors (#17948) · 90d0a74b
  xinli-centml authored May 11, 2025
```
Signed-off-by: Xin Li <xin@centml.ai>
```
  90d0a74b
- [Kernel] fp4 marlin kernel (#17687) · d74e5f37
  Jinzhen Lin authored May 11, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  d74e5f37
10 May, 2025 5 commits
- [v1] Rename specialized_manager.py to single_type_kv_cache_manager.py (#17946) · ca66a167
  Chen Zhang authored May 11, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  ca66a167
- [v1] Pass BlockTable and KVCacheSpec to AttentionMetadataBuilders (#17483) · 950751a9
  Chen Zhang authored May 11, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  950751a9
- Don't default construct `ModelConfig` when default constructing `VllmConfig` (#17943) · 68311891
  Harry Mellor authored May 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  68311891
- Add missing content type headers to /ping and /health (#17036) (#17786) · fc4441a4
  Ximo Guanter authored May 10, 2025
```
Signed-off-by: Ximo Guanter <ximo.guanter@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  fc4441a4
- [V1][Spec Decoding] Log accumulated metrics after system goes idle (#17913) · 7042cc96
  Mark McLoughlin authored May 10, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  7042cc96
09 May, 2025 17 commits
- [Hardware/NVIDIA/Kernel] Enable nvidia/DeepSeek-R1-FP4 Model (#16362) · 0c0fdae8
  Pavani Majety authored May 09, 2025
  
  0c0fdae8
- Improve configs - the rest! (#17562) · 4b2ed792
  Harry Mellor authored May 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  4b2ed792
- [V1][Spec Decoding] Include bonus tokens in mean acceptance length (#17908) · 7e357113
  Mark McLoughlin authored May 09, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  7e357113
- Add option to use torch._inductor.standalone_compile (#17057) · ea2236bf
  Richard Zou authored May 09, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  ea2236bf
- Handle error when `str` passed to `/v1/audio/transcriptions` (#17909) · 7d4aedae
  Harry Mellor authored May 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  7d4aedae
- Update CT WNA16MarlinMoE integration (#16666) · 22481fbf
  Michael Goin authored May 09, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  22481fbf
- [Misc] Auto fallback to float16 for pre-Ampere GPUs when detected bfloat16 config (#17265) · 5c4c08f6
  Isotr0py authored May 10, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  5c4c08f6
- Revert "[BugFix][AMD] Compatible patch for latest AITER(05/07/2025)" (#17910) · 85b72cb7
  Michael Goin authored May 09, 2025
  
  85b72cb7
- [v1] Move block management logic from KVCacheManager to SpecializedManager (#17474) · 200da9a5
  Chen Zhang authored May 09, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  200da9a5
- [BugFix][AMD] Compatible patch for latest AITER(05/07/2025) (#17864) · 9f64e934
  qli88 authored May 09, 2025
```
Signed-off-by: Qiang Li <qiang.li2@amd.com>
```
  9f64e934
- Change `top_k` to be disabled with `0` (still accept `-1` for now) (#17773) · c6798baa
  Harry Mellor authored May 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  c6798baa
- Fix Whisper crash caused by invalid``` max_num_batched_tokens``` config (#17853) · 5b2dcbf0
  inkcherry authored May 09, 2025
```
Signed-off-by: inkcherry <mingzhi.liu@intel.com>
```
  5b2dcbf0
- [Bugfix][CPU] Fix broken AVX2 CPU TP support (#17252) · 6e4a93e3
  Isotr0py authored May 09, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  6e4a93e3
- [Bugfix][ROCm] Fix AITER MLA V1 (#17880) · 217db4ba
  vllmellm authored May 09, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  217db4ba
- [BUGFIX]: return fast when request requires prompt logprobs (#17251) · d310e6de
  Ning Xie authored May 09, 2025
  
  d310e6de
- [Attention] MLA move rotary embedding to cuda-graph region (#17668) · 5e6f9394
  Lucas Wilkinson authored May 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  5e6f9394
- [FEAT][ROCm]: Support AITER MLA on V1 Engine (#17523) · 3c9396a6
  vllmellm authored May 09, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
```
  3c9396a6
08 May, 2025 14 commits
- Add cutlass support for blackwell fp8 blockwise gemm (#14383) · 376786fa
  Shu Wang authored May 08, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
```
  376786fa
- Fix noisy warning for uncalibrated q_scale/p_scale (#17414) · 4f605a6d
  Michael Goin authored May 08, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  4f605a6d
- [V1] Improve VLLM_ALLOW_INSECURE_SERIALIZATION logging (#17860) · 226a4272
  Russell Bryant authored May 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  226a4272
- [Misc] Delete LoRA-related redundancy code (#17841) · a944f8ed
  Jee Jee Li authored May 08, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a944f8ed
- [Bugfix] `use_fast` failing to be propagated to Qwen2-VL image processor (#17838) · 015815fe
  Cyrus Leung authored May 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  015815fe
- [Bugfix] Fix quark fp8 format loading on AMD GPUs (#12612) · bb239a73
  fxmarty-amd authored May 08, 2025
```
Signed-off-by: Felix Marty <felmarty@amd.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
```
  bb239a73
- [TPU] Fix the test_sampler (#17820) · a463555d
  Jevin Jiang authored May 08, 2025
  
  a463555d
- [Misc] support model prefix & add deepseek vl2 tiny fused moe config (#17763) · 0a9bbaa1
  xsank authored May 08, 2025
```
Signed-off-by: 唯勤 <xsank.mz@alibaba-inc.com>
Co-authored-by: 唯勤 <xsank.mz@alibaba-inc.com>
```
  0a9bbaa1
- [Bugfix] Fix bad words for Mistral models (#17753) · 39956efb
  Qiong Zhou Huang authored May 07, 2025
```
Signed-off-by: Qiong Zhou Huang <qiong@phonic.co>
```
  39956efb
- [Qwen3]add qwen3-235b-bf16 fused moe config on A100 (#17715) · 597051e5
  Ximingwang-09 authored May 08, 2025
  
  597051e5
- [Frontend] Chat template fallbacks for multimodal models (#17805) · 96722aa8
  Cyrus Leung authored May 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  96722aa8
- [Hardware][Intel-Gaudi] Support Automatic Prefix Caching on HPU (#17648) · 843b2227
  Agata Dobrzyniewicz authored May 08, 2025
```
Signed-off-by: Agata Dobrzyniewicz <adobrzyniewicz@habana.ai>
```
  843b2227
- [Kernel][Hardware][AMD] Bf16 mfma opt for ROCm skinny GEMMs (#17071) · 5a499e70
  Hashem Hashemi authored May 07, 2025
```
Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: charlifu <charlifu@amd.com>
```
  5a499e70
- [V1] Add VLLM_ALLOW_INSECURE_SERIALIZATION env var (#17490) · 6930a411
  Russell Bryant authored May 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  6930a411