Commits · 53d0cb742366a00acd259beb786ac5d8224bc641 · OpenDAS / vllm_cscc · GitLab

08 May, 2025 21 commits
- [Misc] add chatbox integration (#17828) · 53d0cb74
  Reid authored May 08, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  53d0cb74
- [Easy] Eliminate c10::optional usage in vllm/csrc (#17819) · f50dcb7c
  Lu Fang authored May 08, 2025
  
  f50dcb7c
- [Doc] Fix a typo in the file name (#17836) · a1e19b63
  Cyrus Leung authored May 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  a1e19b63
- [Bugfix] Fix quark fp8 format loading on AMD GPUs (#12612) · bb239a73
  fxmarty-amd authored May 08, 2025
```
Signed-off-by: Felix Marty <felmarty@amd.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
```
  bb239a73
- [TPU] Fix the test_sampler (#17820) · a463555d
  Jevin Jiang authored May 08, 2025
  
  a463555d
- [Bugfix] Fix tool call template validation for Mistral models (#17644) · ca04b97c
  Rick Yuan authored May 08, 2025
```
Signed-off-by: Rick Yuan <yuan821120@gmail.com>
Signed-off-by: RIck Yuan <yuan821120@gmail.com>
Co-authored-by: Aaron Pham <Aaronpham0103@gmail.com>
```
  ca04b97c
- [Misc] support model prefix & add deepseek vl2 tiny fused moe config (#17763) · 0a9bbaa1
  xsank authored May 08, 2025
```
Signed-off-by: 唯勤 <xsank.mz@alibaba-inc.com>
Co-authored-by: 唯勤 <xsank.mz@alibaba-inc.com>
```
  0a9bbaa1
- [Bugfix] Fix bad words for Mistral models (#17753) · 39956efb
  Qiong Zhou Huang authored May 07, 2025
```
Signed-off-by: Qiong Zhou Huang <qiong@phonic.co>
```
  39956efb
- [Qwen3]add qwen3-235b-bf16 fused moe config on A100 (#17715) · 597051e5
  Ximingwang-09 authored May 08, 2025
  
  597051e5
- [Frontend] Chat template fallbacks for multimodal models (#17805) · 96722aa8
  Cyrus Leung authored May 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  96722aa8
- [Hardware][Intel-Gaudi] Support Automatic Prefix Caching on HPU (#17648) · 843b2227
  Agata Dobrzyniewicz authored May 08, 2025
```
Signed-off-by: Agata Dobrzyniewicz <adobrzyniewicz@habana.ai>
```
  843b2227
- [Hardware][Power] Enable compressed tensor W8A8 INT8 quantization for POWER (#17153) · e515668e
  Akash kaothalkar authored May 08, 2025
```
Signed-off-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  e515668e
- [Kernel][Hardware][AMD] Bf16 mfma opt for ROCm skinny GEMMs (#17071) · 5a499e70
  Hashem Hashemi authored May 07, 2025
```
Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: charlifu <charlifu@amd.com>
```
  5a499e70
- [V1] Add VLLM_ALLOW_INSECURE_SERIALIZATION env var (#17490) · 6930a411
  Russell Bryant authored May 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  6930a411
- Only log non-default CLI args for online serving (#17803) · 998eea4a
  Harry Mellor authored May 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  998eea4a
- [Installation] OpenTelemetry version update (#17771) · c747d845
  Mikhail Podvitskii authored May 08, 2025
```
Signed-off-by: Mikhail Podvitskii <podvitskiymichael@gmail.com>
```
  c747d845
- Improve exception reporting in MP engine (#17800) · b2da14a0
  Vadim Markovtsev authored May 08, 2025
```
Signed-off-by: Vadim Markovtsev <vadim@poolside.ai>
```
  b2da14a0
- [Core] Support full cuda graph in v1 (#16072) · 7ea2adb8
  Chanh Nguyen authored May 07, 2025
```
Signed-off-by: Chanh Nguyen <cnguyen@linkedin.com>
Co-authored-by: Chanh Nguyen <cnguyen@linkedin.com>
```
  7ea2adb8
- [BugFix] Fix `--disable-log-stats` in V1 server mode (#17600) · 3d13ca0e
  Nick Hill authored May 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  3d13ca0e
- Don't call the venv `vllm` (#17810) · 66ab3b13
  Harry Mellor authored May 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  66ab3b13
- [Chore][Doc] uses model id determined from OpenAI client (#17815) · a8238bbd
  Aaron Pham authored May 07, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  a8238bbd
07 May, 2025 19 commits
- [Core][Feature] Input metadata dump on crash (#13407) · d43f914d
  Wallas Henrique authored May 07, 2025
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  d43f914d
- [BugFix] Avoid secondary missing `MultiprocExecutor.workers` error (#17811) · ed5272cf
  Nick Hill authored May 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  ed5272cf
- [Hardware][TPU][V1] Multi-LoRA implementation for the V1 TPU backend (#14238) · c20ef40f
  Akshat Tripathi authored May 07, 2025
```
Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
```
  c20ef40f
- [Quantization] Quark MXFP4 format loading (#16943) · db593aa6
  Bowen Bao authored May 07, 2025
  
  db593aa6
- [Bugfix] Fix missing lora name mapping for lora without prefix (#17793) · f98e3075
  Isotr0py authored May 08, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  f98e3075
- Fix and simplify `deprecated=True` CLI `kwarg` (#17781) · 646a31e5
  Harry Mellor authored May 07, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  646a31e5
- [Bugfix] Fix Video IO error for short video (#17791) · be8ff88e
  Isotr0py authored May 07, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  be8ff88e
- Only depend on importlib-metadata for Python < 3.10 (#17776) · 1a6af145
  Christian Heimes authored May 07, 2025
```
Signed-off-by: Christian Heimes <christian@python.org>
```
  1a6af145
- [ROCm][FP8][Kernel] FP8 quantization fused into Custom Paged Attention (#17139) · 32aa74c0
  Gregory Shtrasberg authored May 07, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  32aa74c0
- [doc] update the issue link (#17782) · 7377dd03
  Reid authored May 07, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  7377dd03
- Make key optional for rotary embedding (#17566) · 98c89e16
  Yong Hoon Shin authored May 07, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  98c89e16
- Fix test_memory_usage_no_spec (#17754) · 324a3119
  Yong Hoon Shin authored May 07, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  324a3119
- [Frontend] Add missing chat templates for various MLLMs (#17758) · 8a15c260
  Cyrus Leung authored May 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8a15c260
- Add NeuronxDistributedInference support, Speculative Decoding, Dynamic on-device sampling (#16357) · 043e4c49
  Satyajith Chilappagari authored May 07, 2025
```
Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
Co-authored-by: Aaron Dou <yzdou@amazon.com>
Co-authored-by: Shashwat Srijan <sssrijan@amazon.com>
Co-authored-by: Chongming Ni <chongmni@amazon.com>
Co-authored-by: Amulya Ballakur <amulyaab@amazon.com>
Co-authored-by: Patrick Lange <patlange@amazon.com>
Co-authored-by: Elaine Zhao <elaineyz@amazon.com>
Co-authored-by: Lin Lin Pan <tailinpa@amazon.com>
Co-authored-by: Navyadhara Gogineni <navyadha@amazon.com>
Co-authored-by: Yishan McNabb <yishanm@amazon.com>
Co-authored-by: Mrinal Shukla <181322398+mrinalks@users.noreply.github.com>
```
  043e4c49
- [Misc] Remove qlora_adapter_name_or_path (#17699) · ba7703e6
  Jee Jee Li authored May 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  ba7703e6
- [Kernel] Use fused rmsnorm for some models like qwen3 series (#17735) · f80ae5bd
  Wanrui Dai authored May 07, 2025
```
Signed-off-by: evian <eviantai@u.nus.edu>
Co-authored-by: evian <eviantai@u.nus.edu>
```
  f80ae5bd
- [Kernel] GGUF MoeVec kernel (#16780) · 1a45a613
  Szymon Ożóg authored May 07, 2025
```
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  1a45a613
- [Misc] Use `apply_rotary_emb` from vllm_flash_attn for Qwen2-VL vision RoPE (#17726) · c3e9d506
  Isotr0py authored May 07, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  c3e9d506
- [Misc] Split model loader (#17712) · 822de7fb
  Jee Jee Li authored May 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  822de7fb