Commits · b15fd2be2acae28b25253840588f3eb135a3f709 · OpenDAS / vllm_cscc · GitLab

21 Mar, 2025 9 commits
- [Hardware][TPU] Add check for no additional graph compilation during runtime (#14710) · b15fd2be
  Siyuan Liu authored Mar 20, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  b15fd2be
- [Misc] Better RayExecutor and multiprocessing compatibility (#14705) · 5df2da5b
  Cody Yu authored Mar 20, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  5df2da5b
- [Model] RE: Mamba2 Prefill Performance Tweaks: Fixing Flurry of Unnecessary Memory Copies (#14857) · 296f927f
  Chih-Chieh Yang authored Mar 20, 2025
```
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
```
  296f927f
- [Bugfix] detect alibi and revert to FA2 (#15231) · 0032903a
  Travis Johnson authored Mar 20, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  0032903a
- [V1][TPU] Speed up top-k on TPU by using torch.topk (#15242) · 47195057
  Hyesoo Yang authored Mar 20, 2025
```
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
```
  47195057
- [Bugfix] Fix incorrect qwen2.5-vl attention mask pre-computation (#15200) · 1e508343
  Isotr0py authored Mar 21, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  1e508343
- [Misc] Clean up the BitsAndBytes arguments (#15140) · 10f55fe6
  Jee Jee Li authored Mar 21, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  10f55fe6
- [V1] Scheduler Refactoring [1/N] - Add Scheduler Interface (#15250) · 0c6f5023
  Woosuk Kwon authored Mar 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  0c6f5023
- Enforce that TP > 1 is not supported for Mamba2 if Quantization is Enabled. (#14617) · 06dd0825
  Yu Chin Fabian Lim authored Mar 21, 2025
```
Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>
```
  06dd0825
20 Mar, 2025 15 commits
- [V1] Add flag to disable cascade attention (#15243) · 2b22290c
  Woosuk Kwon authored Mar 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2b22290c
- [Bugfix] fix V1 Engine crash while handling requests with duplicate request id (#15043) · d8e82bc0
  Jason authored Mar 21, 2025
```
Signed-off-by: Jiahui Sun <jhsun2020@gmail.com>
```
  d8e82bc0
- Fix env vars for running Ray distributed backend on GKE (#15166) · a8f12a63
  Richard Liu authored Mar 20, 2025
```
Signed-off-by: Richard Liu <ricliu@google.com>
```
  a8f12a63
- [Bugfix] Multi-video inference on LLaVA-Onevision (#15082) · 27261e40
  Cyrus Leung authored Mar 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  27261e40
- Fixing Imprecise Type Annotations (#15192) · c607a265
  Wang Ran (汪然) authored Mar 20, 2025
  
  c607a265
- [Frontend][Bugfix] support prefill decode disaggregation on deepseek (#14824) · 742369d3
  billishyahao authored Mar 20, 2025
```
Signed-off-by: billishyahao <bill.he@amd.com>
Co-authored-by: Zhai Feiyue <80079571+ZhaiFeiyue@users.noreply.github.com>
```
  742369d3
- typo: Update config.py (#15189) · bfe2fe0a
  Wang Ran (汪然) authored Mar 20, 2025
  
  bfe2fe0a
- Enable CUDA graph support for llama 3.2 vision (#14917) · a8652f4f
  Matt Ritter authored Mar 19, 2025
```
Signed-off-by: Matt Ritter <100659061+mritterfigma@users.noreply.github.com>
```
  a8652f4f
- [Attention] Flash Attention 3 - fp8 (#14570) · a597a575
  Mickaël Seznec authored Mar 20, 2025
```
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
```
  a597a575
- [Misc]fixed disable these http request logs (#14754) · ae65f3e2
  Chauncey authored Mar 20, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  ae65f3e2
- [Core][V0] Add guidance backend for structured output (#14589) · 1f16b7fe
  Russell Bryant authored Mar 20, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <lohuynh@microsoft.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
```
  1f16b7fe
- [V1][TPU] Support V1 Sampler for ragged attention (#14227) · d8c6d7d6
  Nicolò Lucchesi authored Mar 20, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  d8c6d7d6
- [Bugfix] Fix embedding assignment for InternVL-based models (#15086) · ffa443af
  Cyrus Leung authored Mar 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ffa443af
- [BugFix] Lazily import XgrammarBackend to avoid early cuda init (#15171) · c47aafa3
  Nick Hill authored Mar 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  c47aafa3
- [V1] TPU - Tensor parallel MP support (#15059) · cfbca8a2
  Alexander Matveev authored Mar 19, 2025
  
  cfbca8a2
19 Mar, 2025 15 commits
- [FrontEnd][Perf] `merge_async_iterators` fast-path for single-prompt requests (#15150) · 22d33bac
  Nick Hill authored Mar 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  22d33bac
- [V1][TPU] Change kv cache shape. (#15145) · b0e96aae
  iefgnoix authored Mar 19, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  b0e96aae
- simple bugfix: Update stats.py (#15139) · 8310e0b5
  Wang Ran (汪然) authored Mar 20, 2025
  
  8310e0b5
- [FEAT]Support reset prefix cache by specified device (#15003) · 26dd972a
  maobaolong authored Mar 20, 2025
  
  26dd972a
- [Frontend] Remove custom_cache_manager (#13791) · 374ee287
  Alessandro Sangiorgi authored Mar 19, 2025
```
Signed-off-by: fulvius31 <asangior@redhat.com>
```
  374ee287
- [Bugfix] Adjust mllama to regional compilation (#15112) · 8363cd09
  Jan Kaniecki authored Mar 19, 2025
```
Signed-off-by: Jan Kaniecki <jkaniecki@habana.ai>
```
  8363cd09
- [Bugfix] Fix size calculation of processing cache (#15114) · 3d446433
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3d446433
- [Misc] Avoid unnecessary HF `do_rescale` warning when passing dummy data (#15107) · 1fe0fd12
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1fe0fd12
- [V1][Bugfix] Fix oracle for device checking (#15104) · dafb4e50
  Roger Wang authored Mar 19, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  dafb4e50
- [Bugfix] Re-enable Gemma3 for V1 (#14980) · 61f41218
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  61f41218
- [V1] Ensure using int64 for sampled token ids (#15065) · 05ccd0aa
  Woosuk Kwon authored Mar 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  05ccd0aa
- [Core] Update dtype detection and defaults (#14858) · f690372b
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f690372b
- [Model] Remove duplicated message check in Mistral chat completion request (#15069) · 8b3e94a3
  Brayden Zhong authored Mar 19, 2025
```
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  8b3e94a3
- [Model] Pixtral: Remove layer instantiation duplication (#15053) · 437f9162
  Julien Denize authored Mar 19, 2025
```
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
```
  437f9162
- [Misc][V1] Skip device checking if not available (#15061) · 4f065f12
  Cody Yu authored Mar 18, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  4f065f12
18 Mar, 2025 1 commit
- fix long dtype in topk sampling (#15049) · 027827cc
  Chujie Zheng authored Mar 19, 2025
  
  027827cc