Commits · c3fd4d669a4446578ea83bf7c330112339d2d8e1 · OpenDAS / vllm_cscc · GitLab

04 Jun, 2025 16 commits
- [Kernel] Integrate batched/masked deepgemm kernel (#19111) · c3fd4d66
  Varun Sundar Rabindranath authored Jun 04, 2025
```
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>
```
  c3fd4d66
- [Bugfix] fix v1 cpu worker fails on macOS (#19121) · ef3f98b5
  Kebe authored Jun 05, 2025
  
  ef3f98b5
- [Bugfix][EP+DP] Fix internode check (#19112) · d459fae0
  Tyler Michael Smith authored Jun 04, 2025
```
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
```
  d459fae0
- Allow AsyncLLMEngine.generate to target a specific DP rank (#19102) · c8dcc159
  jmswen authored Jun 04, 2025
```
Signed-off-by: Jon Swenson <jmswen@gmail.com>
```
  c8dcc159
- Sm100 blockwise fp8 swap ab (#18564) · 5f2cd251
  Lain authored Jun 04, 2025
  
  5f2cd251
- [CI/Build][Bugfix] Ensure compatibility with transformers 4.52 (#18678) · 01dc9a76
  Cyrus Leung authored Jun 04, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  01dc9a76
- Improve the output precision of embedding models (#19092) · 35cf32df
  wang.yuqi authored Jun 04, 2025
  
  35cf32df
- [Misc] Add packages for benchmark as extra dependency (#19089) · 8711bc5e
  Isotr0py authored Jun 04, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  8711bc5e
- Fix ValueError: Missing value for tag key(s): model_name,engine. (#19113) · 2669a0d7
  Seiji Eicher authored Jun 04, 2025
```
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
```
  2669a0d7
- [Bugfix] Fix FA3 full cuda graph correctness (#19106) · b124e108
  Woosuk Kwon authored Jun 03, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b124e108
- [NVIDIA] Add Cutlass MLA backend (#17625) · 41aa5784
  Kaixi Hou authored Jun 04, 2025
  
  41aa5784
- [KERNEL] Sampler. CUDA kernel for applying repetition penalty (#18437) · 5d6d1adf
  Vadim Gimpelson authored Jun 04, 2025
  
  5d6d1adf
- [Core] Cast multimodal input in hf processor (#18862) · 1409ef91
  Lukas Geiger authored Jun 04, 2025
```
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
```
  1409ef91
- [CPU] V1 support for the CPU backend (#16441) · 4555143e
  Li, Jiang authored Jun 04, 2025
  
  4555143e
- feat: add data parallel rank to KVEventBatch (#18925) · b712be98
  Yan Ru Pei authored Jun 03, 2025
  
  b712be98
- [Bugfix] Max concurrency estimation and check_enough_kv_cache_memory for... · a8da78ea
  Chen Zhang authored Jun 04, 2025
```
[Bugfix] Max concurrency estimation and check_enough_kv_cache_memory for models with sliding window layers (#19029)
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  a8da78ea
03 Jun, 2025 22 commits
- [Bugfix][P/D] Fix Prefix Cache Bug (#18411) · 5d96533e
  Nicolò Lucchesi authored Jun 04, 2025
```
Signed-off-by: nicklucche <nlucches@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
```
  5d96533e
- [Bugfix]: Fix the incompatibility issue with tool_choice 'required' when... · 4de790fc
  Chauncey authored Jun 04, 2025
```
[Bugfix]: Fix the incompatibility issue with tool_choice 'required' when Thinking is enabled (#19075)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  4de790fc
- [Bugfix] get_num_blocks_to_allocate with null_block (#19031) · b5fd9506
  Chen Zhang authored Jun 04, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  b5fd9506
- [V1][Spec Decode][Ngram] 1.35x gain -> 1.95x gain on InstructCoder with prompt fix (#18971) · 135cf55c
  Ekagra Ranjan authored Jun 03, 2025
  
  135cf55c
- [v1] Re-init input batch for multiple kv cache groups (#18654) · 6cac54f4
  Chen Zhang authored Jun 04, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  6cac54f4
- Fix interaction between `Optional` and `Annotated` in CLI typing (#19093) · 6865fe00
  Harry Mellor authored Jun 03, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Yikun Jiang <yikun@apache.org>
```
  6865fe00
- [V1] Support cross-layer KV sharing (#18212) · bdf13965
  Yong Hoon Shin authored Jun 03, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  bdf13965
- [Kernel] DeepEP dispatch-combine kernel integration (#18434) · fa98d777
  Varun Sundar Rabindranath authored Jun 03, 2025
```
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
  fa98d777
- [Misc] Add SPDX-FileCopyrightText (#19100) · 02f0c7b2
  Simon Mo authored Jun 03, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  02f0c7b2
- [Misc] fix: add miss best_of param validation (#18555) · d054da19
  CYJiang authored Jun 04, 2025
```
Signed-off-by: googs1025 <googs1025@gmail.com>
```
  d054da19
- [Misc] Add missing `_Backend` enums (#19081) · 4b7817c1
  Nicolò Lucchesi authored Jun 03, 2025
```
Signed-off-by: nicklucche <nlucches@redhat.com>
```
  4b7817c1
- [Bugfix] disable processor cache (#19068) · d81edded
  Raushan Turganbay authored Jun 03, 2025
```
Signed-off-by: raushan <raushan@huggingface.co>
```
  d81edded
- Fix underscores in dict keys passed via CLI (#19030) · 476844d4
  Harry Mellor authored Jun 03, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  476844d4
- [Misc] Update `WeightsMapper` for qwen2-vl/qwen2.5-vl (#19054) · ec2dcd80
  Isotr0py authored Jun 03, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  ec2dcd80
- [v1][KVCacheManager] Rename BlockHashType to BlockHash (#19015) · f32fcd94
  Chen Zhang authored Jun 03, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  f32fcd94
- Reduce logs in CLI scripts and plugin loader (#18970) · cc977286
  Michael Goin authored Jun 03, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  cc977286
- [bugfix] small fix logic issue (#18999) · 17430e36
  Reid authored Jun 03, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  17430e36
- Add tarsier model support (#18985) · 1282bd81
  汪志鹏 authored Jun 03, 2025
```
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
```
  1282bd81
- [V1] Support DP with Ray (#18779) · bdce64f2
  Rui Qiao authored Jun 02, 2025
  
  bdce64f2
- [CPU][CI] Re-enable the CPU CI tests (#19046) · 8655f47f
  Li, Jiang authored Jun 03, 2025
```
Signed-off-by: jiang.li <jiang1.li@intel.com>
```
  8655f47f
- [Bugfix][EP+DP] Use pplx-kernel internode instead of intranode (#19034) · 8a57872b
  Tyler Michael Smith authored Jun 02, 2025
```
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  8a57872b
- [Hardware][TPU] Initial support of model parallelism with single worker using SPMD (#18011) · 9112b443
  Siyuan Liu authored Jun 02, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Co-authored-by: Hossein Sarshar <hossein.sarshar@gmail.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
```
  9112b443
02 Jun, 2025 2 commits
- [Bugfix][Model] Attempt to fix eagle in V0. (#18978) · ca2f6b9c
  Gregory Shtrasberg authored Jun 02, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  ca2f6b9c
- [Frontend] enable custom logging for the uvicorn server (OpenAI API server) (#18403) · 20133cfe
  Frαnçois authored Jun 02, 2025
```
Signed-off-by: François Paupier <francois.paupier@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  20133cfe