Commits · 2b22290ce01b033cc692e7dce159d74a43f6f2c5 · OpenDAS / vllm_cscc · GitLab

20 Mar, 2025 15 commits
- [V1] Add flag to disable cascade attention (#15243) · 2b22290c
  Woosuk Kwon authored Mar 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2b22290c
- [Bugfix] fix V1 Engine crash while handling requests with duplicate request id (#15043) · d8e82bc0
  Jason authored Mar 21, 2025
```
Signed-off-by: Jiahui Sun <jhsun2020@gmail.com>
```
  d8e82bc0
- Fix env vars for running Ray distributed backend on GKE (#15166) · a8f12a63
  Richard Liu authored Mar 20, 2025
```
Signed-off-by: Richard Liu <ricliu@google.com>
```
  a8f12a63
- [Bugfix] Multi-video inference on LLaVA-Onevision (#15082) · 27261e40
  Cyrus Leung authored Mar 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  27261e40
- Fixing Imprecise Type Annotations (#15192) · c607a265
  Wang Ran (汪然) authored Mar 20, 2025
  
  c607a265
- [Frontend][Bugfix] support prefill decode disaggregation on deepseek (#14824) · 742369d3
  billishyahao authored Mar 20, 2025
```
Signed-off-by: billishyahao <bill.he@amd.com>
Co-authored-by: Zhai Feiyue <80079571+ZhaiFeiyue@users.noreply.github.com>
```
  742369d3
- typo: Update config.py (#15189) · bfe2fe0a
  Wang Ran (汪然) authored Mar 20, 2025
  
  bfe2fe0a
- Enable CUDA graph support for llama 3.2 vision (#14917) · a8652f4f
  Matt Ritter authored Mar 19, 2025
```
Signed-off-by: Matt Ritter <100659061+mritterfigma@users.noreply.github.com>
```
  a8652f4f
- [Attention] Flash Attention 3 - fp8 (#14570) · a597a575
  Mickaël Seznec authored Mar 20, 2025
```
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
```
  a597a575
- [Misc]fixed disable these http request logs (#14754) · ae65f3e2
  Chauncey authored Mar 20, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  ae65f3e2
- [Core][V0] Add guidance backend for structured output (#14589) · 1f16b7fe
  Russell Bryant authored Mar 20, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <lohuynh@microsoft.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
```
  1f16b7fe
- [V1][TPU] Support V1 Sampler for ragged attention (#14227) · d8c6d7d6
  Nicolò Lucchesi authored Mar 20, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  d8c6d7d6
- [Bugfix] Fix embedding assignment for InternVL-based models (#15086) · ffa443af
  Cyrus Leung authored Mar 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ffa443af
- [BugFix] Lazily import XgrammarBackend to avoid early cuda init (#15171) · c47aafa3
  Nick Hill authored Mar 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  c47aafa3
- [V1] TPU - Tensor parallel MP support (#15059) · cfbca8a2
  Alexander Matveev authored Mar 19, 2025
  
  cfbca8a2
19 Mar, 2025 15 commits
- [FrontEnd][Perf] `merge_async_iterators` fast-path for single-prompt requests (#15150) · 22d33bac
  Nick Hill authored Mar 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  22d33bac
- [V1][TPU] Change kv cache shape. (#15145) · b0e96aae
  iefgnoix authored Mar 19, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  b0e96aae
- simple bugfix: Update stats.py (#15139) · 8310e0b5
  Wang Ran (汪然) authored Mar 20, 2025
  
  8310e0b5
- [FEAT]Support reset prefix cache by specified device (#15003) · 26dd972a
  maobaolong authored Mar 20, 2025
  
  26dd972a
- [Frontend] Remove custom_cache_manager (#13791) · 374ee287
  Alessandro Sangiorgi authored Mar 19, 2025
```
Signed-off-by: fulvius31 <asangior@redhat.com>
```
  374ee287
- [Bugfix] Adjust mllama to regional compilation (#15112) · 8363cd09
  Jan Kaniecki authored Mar 19, 2025
```
Signed-off-by: Jan Kaniecki <jkaniecki@habana.ai>
```
  8363cd09
- [Bugfix] Fix size calculation of processing cache (#15114) · 3d446433
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3d446433
- [Misc] Avoid unnecessary HF `do_rescale` warning when passing dummy data (#15107) · 1fe0fd12
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1fe0fd12
- [V1][Bugfix] Fix oracle for device checking (#15104) · dafb4e50
  Roger Wang authored Mar 19, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  dafb4e50
- [Bugfix] Re-enable Gemma3 for V1 (#14980) · 61f41218
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  61f41218
- [V1] Ensure using int64 for sampled token ids (#15065) · 05ccd0aa
  Woosuk Kwon authored Mar 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  05ccd0aa
- [Core] Update dtype detection and defaults (#14858) · f690372b
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f690372b
- [Model] Remove duplicated message check in Mistral chat completion request (#15069) · 8b3e94a3
  Brayden Zhong authored Mar 19, 2025
```
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  8b3e94a3
- [Model] Pixtral: Remove layer instantiation duplication (#15053) · 437f9162
  Julien Denize authored Mar 19, 2025
```
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
```
  437f9162
- [Misc][V1] Skip device checking if not available (#15061) · 4f065f12
  Cody Yu authored Mar 18, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  4f065f12
18 Mar, 2025 10 commits
- fix long dtype in topk sampling (#15049) · 027827cc
  Chujie Zheng authored Mar 19, 2025
  
  027827cc
- [V1][Spec Decode] Optimize Rejection Sampler with Triton Kernels (#14930) · 99abb8b6
  Woosuk Kwon authored Mar 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  99abb8b6
- [V1] Refactor Structured Output for multiple backends (#14694) · 3a1e6481
  Russell Bryant authored Mar 18, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  3a1e6481
- [Bugfix] Fix LoRA extra vocab size (#15047) · 46c759c1
  Jee Jee Li authored Mar 19, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  46c759c1
- [Bugfix] Fix broken CPU quantization due to triton import (#15038) · 179a619c
  Isotr0py authored Mar 18, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  179a619c
- [MODEL] Add support for Zamba2 models (#13185) · 452e8fd9
  yury-tokpanov authored Mar 18, 2025
```
Signed-off-by: Yury Tokpanov <yury@zyphra.com>
Signed-off-by: Quentin Anthony <qganthony@yahoo.com>
Co-authored-by: Quentin Anthony <qganthony@yahoo.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  452e8fd9
- MI325 configs, fused_moe_kernel bugfix (#14987) · 8b793f7e
  ekuznetsov139 authored Mar 18, 2025
```
Signed-off-by: Eugene Kuznetsov <eugene.kuznetsov@amd.com>
```
  8b793f7e
- [TPU][V1][Bugfix] Fix chunked prefill with padding (#15037) · af35d3a3
  Nicolò Lucchesi authored Mar 18, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  af35d3a3
- [Bugfix] Register serializers for V0 MQ Engine (#15009) · 3b457143
  Simon Mo authored Mar 18, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  3b457143
- [Bugfix] Loosen type check to avoid errors in V1 (#15021) · ab656f2c
  Cyrus Leung authored Mar 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ab656f2c