Commits · c8dcc159214a20650451dcd64b226f56671763f1 · OpenDAS / vllm_cscc

"cacheflow/frontend/simple_frontend.py" did not exist on "1132fae0ca7f6103b0dd7b96932c75f5ed780288"

04 Jun, 2025 11 commits
- Allow AsyncLLMEngine.generate to target a specific DP rank (#19102) · c8dcc159
  jmswen authored Jun 04, 2025
```
Signed-off-by: Jon Swenson <jmswen@gmail.com>
```
  c8dcc159
- [CI/Build][Bugfix] Ensure compatibility with transformers 4.52 (#18678) · 01dc9a76
  Cyrus Leung authored Jun 04, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  01dc9a76
- Improve the output precision of embedding models (#19092) · 35cf32df
  wang.yuqi authored Jun 04, 2025
  
  35cf32df
- Fix ValueError: Missing value for tag key(s): model_name,engine. (#19113) · 2669a0d7
  Seiji Eicher authored Jun 04, 2025
```
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
```
  2669a0d7
- [TPU] Skip hanging tests (#19115) · 8e972d9c
  Siyuan Liu authored Jun 04, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  8e972d9c
- [Bugfix] Fix FA3 full cuda graph correctness (#19106) · b124e108
  Woosuk Kwon authored Jun 03, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b124e108
- [NVIDIA] Add Cutlass MLA backend (#17625) · 41aa5784
  Kaixi Hou authored Jun 04, 2025
  
  41aa5784
- [KERNEL] Sampler. CUDA kernel for applying repetition penalty (#18437) · 5d6d1adf
  Vadim Gimpelson authored Jun 04, 2025
  
  5d6d1adf
- [CPU] V1 support for the CPU backend (#16441) · 4555143e
  Li, Jiang authored Jun 04, 2025
  
  4555143e
- feat: add data parallel rank to KVEventBatch (#18925) · b712be98
  Yan Ru Pei authored Jun 03, 2025
  
  b712be98
- [Bugfix] Max concurrency estimation and check_enough_kv_cache_memory for... · a8da78ea
  Chen Zhang authored Jun 04, 2025
```
[Bugfix] Max concurrency estimation and check_enough_kv_cache_memory for models with sliding window layers (#19029)
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  a8da78ea
03 Jun, 2025 13 commits
- [Bugfix]: Fix the incompatibility issue with tool_choice 'required' when... · 4de790fc
  Chauncey authored Jun 04, 2025
```
[Bugfix]: Fix the incompatibility issue with tool_choice 'required' when Thinking is enabled (#19075)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  4de790fc
- [Bugfix] get_num_blocks_to_allocate with null_block (#19031) · b5fd9506
  Chen Zhang authored Jun 04, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  b5fd9506
- [v1] Re-init input batch for multiple kv cache groups (#18654) · 6cac54f4
  Chen Zhang authored Jun 04, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  6cac54f4
- Fix interaction between `Optional` and `Annotated` in CLI typing (#19093) · 6865fe00
  Harry Mellor authored Jun 03, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Yikun Jiang <yikun@apache.org>
```
  6865fe00
- [V1] Support cross-layer KV sharing (#18212) · bdf13965
  Yong Hoon Shin authored Jun 03, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  bdf13965
- [Kernel] DeepEP dispatch-combine kernel integration (#18434) · fa98d777
  Varun Sundar Rabindranath authored Jun 03, 2025
```
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
  fa98d777
- [Misc] Add SPDX-FileCopyrightText (#19100) · 02f0c7b2
  Simon Mo authored Jun 03, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  02f0c7b2
- Fix underscores in dict keys passed via CLI (#19030) · 476844d4
  Harry Mellor authored Jun 03, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  476844d4
- [CI/Build] Remove V0 LoRA test (#19066) · 4e68ae5e
  Jee Jee Li authored Jun 03, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  4e68ae5e
- [v1][KVCacheManager] Rename BlockHashType to BlockHash (#19015) · f32fcd94
  Chen Zhang authored Jun 03, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  f32fcd94
- Add tarsier model support (#18985) · 1282bd81
  汪志鹏 authored Jun 03, 2025
```
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
```
  1282bd81
- [V1] Support DP with Ray (#18779) · bdce64f2
  Rui Qiao authored Jun 02, 2025
  
  bdce64f2
- [Hardware][TPU] Initial support of model parallelism with single worker using SPMD (#18011) · 9112b443
  Siyuan Liu authored Jun 02, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Co-authored-by: Hossein Sarshar <hossein.sarshar@gmail.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
```
  9112b443
02 Jun, 2025 1 commit
- [Core] Support inplace model weights loading (#18745) · 9760fd8f
  22quinn authored Jun 02, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  9760fd8f
01 Jun, 2025 1 commit
- [Core] Rework dtype resolution (#18751) · 6aa8f9a4
  Cyrus Leung authored Jun 01, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6aa8f9a4
31 May, 2025 4 commits
- [Misc] add return token strs for tokenize (#18941) · 20079c6e
  Reid authored Jun 01, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  20079c6e
- [ROCm][Kernel] Add gfx950 support for skinny gemms (#18010) · 306d6040
  Charlie Fu authored May 31, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
```
  306d6040
- [FEAT][ROCm] Add AITER grouped topk for DeepSeekV2 (#18825) · 0f5e0d56
  vllmellm authored May 31, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  0f5e0d56
- [Frontend] Add rerank support to run_batch endpoint (#16278) · dff80b0e
  Pooya Davoodi authored May 31, 2025
```
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
```
  dff80b0e
30 May, 2025 6 commits
- Tool parser regex timeout handling (#18960) · 1dab4d57
  Will Eaton authored May 30, 2025
```
Signed-off-by: Will Eaton <weaton@redhat.com>
```
  1dab4d57
- [VLM] Add PP support and fix GPTQ inference for Ovis models (#18958) · 5a864163
  Isotr0py authored May 31, 2025
```
Signed-off-by: isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  5a864163
- [Perf] API-server scaleout with many-to-many server-engine comms (#17546) · 2dbe8c07
  Nick Hill authored May 30, 2025
  
  2dbe8c07
- [Feature] minicpm eagle support (#18943) · e1fadf11
  Shawn Huang authored May 30, 2025
```
Signed-off-by: huangyuxiang03 <huangyx0321@gmail.com>
Co-authored-by: huangyuxiang03 <huangyx0321@gmail.com>
```
  e1fadf11
- [Bugfix][TPU] Fix tpu model runner testcase failure (#18810) · fba02e3b
  Carol Zheng authored May 30, 2025
```
Signed-off-by: Carol Zheng <cazheng@google.com>
```
  fba02e3b
- [Misc] Update type annotation for rotary embedding `base` (#18914) · 1aa2f81b
  Cyrus Leung authored May 30, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1aa2f81b
29 May, 2025 4 commits
- [TPU] remove transpose ops in moe kernel (#18923) · a1cc9f33
  Chengji Yao authored May 29, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  a1cc9f33
- [BugFix] Make DP work with connector-delayed new requests (#18559) · d1d61f33
  Nick Hill authored May 29, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Will Eaton <weaton@redhat.com>
```
  d1d61f33
- [V1] Allocate kv_cache with stride order for V1 (#18775) · 32ce3cf7
  Nicolò Lucchesi authored May 29, 2025
```
Signed-off-by: nicklucche <nlucches@redhat.com>
```
  32ce3cf7
- [Deprecation] Disallow pos-args other than `model` when initializing `LLM` (#18802) · c2903403
  Cyrus Leung authored May 30, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c2903403