Commits · 7d4aedae7c3e1ff4a1b1dea1f8df3257a7878d0a · OpenDAS / vllm_cscc · GitLab

09 May, 2025 20 commits
- Handle error when `str` passed to `/v1/audio/transcriptions` (#17909) · 7d4aedae
  Harry Mellor authored May 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  7d4aedae
- Update CT WNA16MarlinMoE integration (#16666) · 22481fbf
  Michael Goin authored May 09, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  22481fbf
- [Misc] Auto fallback to float16 for pre-Ampere GPUs when detected bfloat16 config (#17265) · 5c4c08f6
  Isotr0py authored May 10, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  5c4c08f6
- [Misc] Add references in ray_serve_deepseek example (#17907) · c44c384b
  Rui Qiao authored May 09, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  c44c384b
- Revert "[BugFix][AMD] Compatible patch for latest AITER(05/07/2025)" (#17910) · 85b72cb7
  Michael Goin authored May 09, 2025
  
  85b72cb7
- [CI/Build] Automatically retry flaky tests (#17856) · 6e5595ca
  Cyrus Leung authored May 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6e5595ca
- [v1] Move block management logic from KVCacheManager to SpecializedManager (#17474) · 200da9a5
  Chen Zhang authored May 09, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  200da9a5
- [BugFix][AMD] Compatible patch for latest AITER(05/07/2025) (#17864) · 9f64e934
  qli88 authored May 09, 2025
```
Signed-off-by: Qiang Li <qiang.li2@amd.com>
```
  9f64e934
- [Misc] add dify integration (#17895) · ec61ea20
  Reid authored May 09, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  ec61ea20
- Change `top_k` to be disabled with `0` (still accept `-1` for now) (#17773) · c6798baa
  Harry Mellor authored May 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  c6798baa
- Fix Whisper crash caused by invalid``` max_num_batched_tokens``` config (#17853) · 5b2dcbf0
  inkcherry authored May 09, 2025
```
Signed-off-by: inkcherry <mingzhi.liu@intel.com>
```
  5b2dcbf0
- [Bugfix][CPU] Fix broken AVX2 CPU TP support (#17252) · 6e4a93e3
  Isotr0py authored May 09, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  6e4a93e3
- [Bugfix][ROCm] Fix AITER MLA V1 (#17880) · 217db4ba
  vllmellm authored May 09, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  217db4ba
- [Doc] remove visible token in doc (#17884) · ff8c4005
  Yan Ma authored May 09, 2025
```
Signed-off-by: yan <yanma1@habana.ai>
```
  ff8c4005
- [Doc] Update several links in reasoning_outputs.md (#17846) · 89a0315f
  Michael Yao authored May 09, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  89a0315f
- [Docs] Add Slides from NYC Meetup (#17879) · 3d1e3876
  Simon Mo authored May 08, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  3d1e3876
- [BUGFIX]: return fast when request requires prompt logprobs (#17251) · d310e6de
  Ning Xie authored May 09, 2025
  
  d310e6de
- [Attention] MLA move rotary embedding to cuda-graph region (#17668) · 5e6f9394
  Lucas Wilkinson authored May 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  5e6f9394
- [V1][Structured Output] Update llguidance (`>= 0.7.11`) to avoid... · 760e3ecc
  Shanshan Shen authored May 09, 2025
```
[V1][Structured Output] Update llguidance (`>= 0.7.11`) to avoid AttributeError (no `StructTag`)  (#17839)
Signed-off-by: shen-shanshan <467638484@qq.com>
```
  760e3ecc
- [FEAT][ROCm]: Support AITER MLA on V1 Engine (#17523) · 3c9396a6
  vllmellm authored May 09, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
```
  3c9396a6
08 May, 2025 20 commits
- Add cutlass support for blackwell fp8 blockwise gemm (#14383) · 376786fa
  Shu Wang authored May 08, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
```
  376786fa
- Fix noisy warning for uncalibrated q_scale/p_scale (#17414) · 4f605a6d
  Michael Goin authored May 08, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  4f605a6d
- [CI] Prune down lm-eval small tests (#17012) · 8342e3ab
  Michael Goin authored May 08, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  8342e3ab
- [Test] Attempt all TPU V1 tests, even if some of them fail. (#17334) · a83a0f92
  yarongmu-google authored May 08, 2025
```
Signed-off-by: Yarong Mu <ymu@google.com>
```
  a83a0f92
- [V1] Improve VLLM_ALLOW_INSECURE_SERIALIZATION logging (#17860) · 226a4272
  Russell Bryant authored May 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  226a4272
- [CI] Fix test_collective_rpc (#17858) · ec54d73c
  Russell Bryant authored May 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  ec54d73c
- [Misc] Delete LoRA-related redundancy code (#17841) · a944f8ed
  Jee Jee Li authored May 08, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a944f8ed
- [Bugfix] `use_fast` failing to be propagated to Qwen2-VL image processor (#17838) · 015815fe
  Cyrus Leung authored May 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  015815fe
- Fix transient dependency error in docs build (#17848) · e4ca6e3a
  Harry Mellor authored May 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e4ca6e3a
- [Misc] add chatbox integration (#17828) · 53d0cb74
  Reid authored May 08, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  53d0cb74
- [Easy] Eliminate c10::optional usage in vllm/csrc (#17819) · f50dcb7c
  Lu Fang authored May 08, 2025
  
  f50dcb7c
- [Doc] Fix a typo in the file name (#17836) · a1e19b63
  Cyrus Leung authored May 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  a1e19b63
- [Bugfix] Fix quark fp8 format loading on AMD GPUs (#12612) · bb239a73
  fxmarty-amd authored May 08, 2025
```
Signed-off-by: Felix Marty <felmarty@amd.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
```
  bb239a73
- [TPU] Fix the test_sampler (#17820) · a463555d
  Jevin Jiang authored May 08, 2025
  
  a463555d
- [Bugfix] Fix tool call template validation for Mistral models (#17644) · ca04b97c
  Rick Yuan authored May 08, 2025
```
Signed-off-by: Rick Yuan <yuan821120@gmail.com>
Signed-off-by: RIck Yuan <yuan821120@gmail.com>
Co-authored-by: Aaron Pham <Aaronpham0103@gmail.com>
```
  ca04b97c
- [Misc] support model prefix & add deepseek vl2 tiny fused moe config (#17763) · 0a9bbaa1
  xsank authored May 08, 2025
```
Signed-off-by: 唯勤 <xsank.mz@alibaba-inc.com>
Co-authored-by: 唯勤 <xsank.mz@alibaba-inc.com>
```
  0a9bbaa1
- [Bugfix] Fix bad words for Mistral models (#17753) · 39956efb
  Qiong Zhou Huang authored May 07, 2025
```
Signed-off-by: Qiong Zhou Huang <qiong@phonic.co>
```
  39956efb
- [Qwen3]add qwen3-235b-bf16 fused moe config on A100 (#17715) · 597051e5
  Ximingwang-09 authored May 08, 2025
  
  597051e5
- [Frontend] Chat template fallbacks for multimodal models (#17805) · 96722aa8
  Cyrus Leung authored May 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  96722aa8
- [Hardware][Intel-Gaudi] Support Automatic Prefix Caching on HPU (#17648) · 843b2227
  Agata Dobrzyniewicz authored May 08, 2025
```
Signed-off-by: Agata Dobrzyniewicz <adobrzyniewicz@habana.ai>
```
  843b2227