Commits · a8f12a63fde4765dffe53f7bf1482d52ac80af33 · OpenDAS / vllm_cscc · GitLab

20 Mar, 2025 22 commits
- Fix env vars for running Ray distributed backend on GKE (#15166) · a8f12a63
  Richard Liu authored Mar 20, 2025
```
Signed-off-by: Richard Liu <ricliu@google.com>
```
  a8f12a63
- Add user forum to README (#15220) · 69ae2380
  Harry Mellor authored Mar 20, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  69ae2380
- [Bugfix] Multi-video inference on LLaVA-Onevision (#15082) · 27261e40
  Cyrus Leung authored Mar 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  27261e40
- [macOS] Ugrade pytorch to 2.6.0 (#15129) · e3f813c3
  Quang-Linh LE authored Mar 20, 2025
  
  e3f813c3
- Fixing Imprecise Type Annotations (#15192) · c607a265
  Wang Ran (汪然) authored Mar 20, 2025
  
  c607a265
- [release] Tag vllm-cpu with latest upon new version released (#15193) · 3d45e3d7
  Kevin H. Luu authored Mar 20, 2025
  
  3d45e3d7
- [Frontend][Bugfix] support prefill decode disaggregation on deepseek (#14824) · 742369d3
  billishyahao authored Mar 20, 2025
```
Signed-off-by: billishyahao <bill.he@amd.com>
Co-authored-by: Zhai Feiyue <80079571+ZhaiFeiyue@users.noreply.github.com>
```
  742369d3
- typo: Update config.py (#15189) · bfe2fe0a
  Wang Ran (汪然) authored Mar 20, 2025
  
  bfe2fe0a
- Enable CUDA graph support for llama 3.2 vision (#14917) · a8652f4f
  Matt Ritter authored Mar 19, 2025
```
Signed-off-by: Matt Ritter <100659061+mritterfigma@users.noreply.github.com>
```
  a8652f4f
- [Doc] Update README.md (#15187) · 2f726b24
  Cyrus Leung authored Mar 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  2f726b24
- [Attention] Flash Attention 3 - fp8 (#14570) · a597a575
  Mickaël Seznec authored Mar 20, 2025
```
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
```
  a597a575
- [Misc]fixed disable these http request logs (#14754) · ae65f3e2
  Chauncey authored Mar 20, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  ae65f3e2
- [Doc] Update Mistral Small 3.1/Pixtral example (#15184) · 34868b10
  Roger Wang authored Mar 19, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  34868b10
- [Core][V0] Add guidance backend for structured output (#14589) · 1f16b7fe
  Russell Bryant authored Mar 20, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <lohuynh@microsoft.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
```
  1f16b7fe
- [Benchmark] Allow oversample request in benchmark dataset (#15170) · b88be221
  Jennifer Zhao authored Mar 19, 2025
```
Signed-off-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
```
  b88be221
- [V1][TPU] Support V1 Sampler for ragged attention (#14227) · d8c6d7d6
  Nicolò Lucchesi authored Mar 20, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  d8c6d7d6
- fix "Total generated tokens:" is 0 if using --backend tgi and --endpo… (#14673) · 40828ce5
  Wang, Yi authored Mar 20, 2025
```
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
```
  40828ce5
- [Bugfix] Fix embedding assignment for InternVL-based models (#15086) · ffa443af
  Cyrus Leung authored Mar 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ffa443af
- Fix broken tests (#14713) · 70e500ca
  Jovan Sardinha authored Mar 19, 2025
```
Signed-off-by: JovanSardinha <jovan.sardinha@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  70e500ca
- [Doc] Clarify run vllm only on one node in distributed inference (#15148) · 4cb1c05c
  Rui Qiao authored Mar 19, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  4cb1c05c
- [BugFix] Lazily import XgrammarBackend to avoid early cuda init (#15171) · c47aafa3
  Nick Hill authored Mar 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  c47aafa3
- [V1] TPU - Tensor parallel MP support (#15059) · cfbca8a2
  Alexander Matveev authored Mar 19, 2025
  
  cfbca8a2
19 Mar, 2025 18 commits
- [Docs] Annouce Ollama and Singapore Meetups (#15161) · 0fe56098
  Simon Mo authored Mar 19, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  0fe56098
- [FrontEnd][Perf] `merge_async_iterators` fast-path for single-prompt requests (#15150) · 22d33bac
  Nick Hill authored Mar 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  22d33bac
- [V1][TPU] Change kv cache shape. (#15145) · b0e96aae
  iefgnoix authored Mar 19, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  b0e96aae
- simple bugfix: Update stats.py (#15139) · 8310e0b5
  Wang Ran (汪然) authored Mar 20, 2025
  
  8310e0b5
- [FEAT]Support reset prefix cache by specified device (#15003) · 26dd972a
  maobaolong authored Mar 20, 2025
  
  26dd972a
- [V1] Minor V1 async engine test refactor (#15075) · 61c7a1b8
  Murali Andoorveedu authored Mar 19, 2025
```
Signed-off-by: andoorve <murali.andoorveedu@mail.utoronto.ca>
Co-authored-by: andoorve <murali.andoorveedu@mail.utoronto.ca>
```
  61c7a1b8
- [Frontend] Remove custom_cache_manager (#13791) · 374ee287
  Alessandro Sangiorgi authored Mar 19, 2025
```
Signed-off-by: fulvius31 <asangior@redhat.com>
```
  374ee287
- [Misc] Update the "the first vLLM China Meetup" slides link to point to the first page (#15134) · a4d83661
  Kero Liang authored Mar 19, 2025
```
Signed-off-by: imkero <kerorek@outlook.com>
```
  a4d83661
- [Bugfix] Adjust mllama to regional compilation (#15112) · 8363cd09
  Jan Kaniecki authored Mar 19, 2025
```
Signed-off-by: Jan Kaniecki <jkaniecki@habana.ai>
```
  8363cd09
- [Misc][Benchmark] Add support for different `tokenizer_mode` (#15040) · 6c5a3195
  Aaron Pham authored Mar 19, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
```
  6c5a3195
- [Doc] Update tip info on using latest transformers when creating a custom Dockerfile (#15070) · 073d1ed3
  Marc-Alexandre Côté authored Mar 19, 2025
  
  073d1ed3
- [Bugfix] Fix size calculation of processing cache (#15114) · 3d446433
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3d446433
- [Misc] Avoid unnecessary HF `do_rescale` warning when passing dummy data (#15107) · 1fe0fd12
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1fe0fd12
- [V1][Bugfix] Fix oracle for device checking (#15104) · dafb4e50
  Roger Wang authored Mar 19, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  dafb4e50
- [CI][Intel GPU] update XPU dockerfile and CI script (#15109) · 68cf1601
  Kunshang Ji authored Mar 19, 2025
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  68cf1601
- [Bugfix] Re-enable Gemma3 for V1 (#14980) · 61f41218
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  61f41218
- [V1] Ensure using int64 for sampled token ids (#15065) · 05ccd0aa
  Woosuk Kwon authored Mar 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  05ccd0aa
- [Core] Update dtype detection and defaults (#14858) · f690372b
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f690372b