Commits · 4f6593b058dc7ba66d887442ba5763c6c1b3886e · OpenDAS / vllm_cscc

11 Sep, 2025 34 commits
- [HybridKVCache][Platform] Add support_hybrid_kv_cache for platform (#24646) · 4f6593b0
  Mengqing Cao authored Sep 11, 2025
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
  4f6593b0
- Allow users to specify kv cache memory size (#21489) · 94e6b2d5
  Boyuan Feng authored Sep 11, 2025
```
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  94e6b2d5
- [CI] Split mteb test from Language Models Test (#24634) · fd1ce98c
  wang.yuqi authored Sep 11, 2025
```
Signed-off-by: wang.yuqi <noooop@126.com>
```
  fd1ce98c
- [Bench] Add qwen-next in benchmark_moe.py (#24661) · d11ec124
  Jee Jee Li authored Sep 11, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  d11ec124
- [build] add torch to tool.uv no-build-isolation-package (#24303) · f5107158
  youkaichao authored Sep 11, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  f5107158
- [Docs] Fixes a typo in the qwen3next model name. (#24654) · f9461974
  Tao He authored Sep 11, 2025
```
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
```
  f9461974
- [XPU] add missing dependency tblib for XPU CI (#24639) · 0cd72a7b
  Fanli Lin authored Sep 11, 2025
```
Signed-off-by: Fanli Lin <fanli.lin@intel.com>
```
  0cd72a7b
- Move `LoRAConfig` from `config/__init__.py` to `config/lora.py` (#24644) · 5f5271f1
  Harry Mellor authored Sep 11, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  5f5271f1
- Fix typing for `safetensors_load_strategy` (#24641) · d6249d06
  Harry Mellor authored Sep 11, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6249d06
- [CI Failure] fix models/language/pooling/test_auto_prefix_cache_support.py (#24636) · 25bb9e8c
  wang.yuqi authored Sep 11, 2025
```
Signed-off-by: wang.yuqi <noooop@126.com>
```
  25bb9e8c
- [Misc] Add @NickLucche to codeowners (#24647) · a1213fae
  Nicolò Lucchesi authored Sep 11, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  a1213fae
- [CI] Split pooling from entrypoints Test (#24632) · a8b0361c
  wang.yuqi authored Sep 11, 2025
```
Signed-off-by: wang.yuqi <noooop@126.com>
```
  a8b0361c
- [Bugfix] Fix _synced_weight_loader (#24565) · ed5ae4aa
  Kyuyeun Kim authored Sep 11, 2025
```
Signed-off-by: Kyuyeun Kim <kyuyeunk@google.com>
```
  ed5ae4aa
- [CI]Add transformers_utils to Async Engine, Inputs, Utils, Worker Test (#24615) · 0fc36463
  Xingyu Liu authored Sep 11, 2025
```
Signed-off-by: Xingyu Liu <charlotteliu12x@gmail.com>
```
  0fc36463
- [Docs] Use 1-2-3 list for deploy steps in deployment/frameworks/ (#24633) · d14c4ebf
  Michael Yao authored Sep 11, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  d14c4ebf
- [Docs] Update V1 doc to reflect whisper support (#24606) · ba601102
  Russell Bryant authored Sep 11, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  ba601102
- [Docs] Revise frameworks/anything-llm.md (#24489) · 85df8afd
  Michael Yao authored Sep 11, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  85df8afd
- [Bugfix] Fix incorrect import of CacheConfig (#24631) · 6aeb1dab
  Cyrus Leung authored Sep 11, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6aeb1dab
- Add the support for the qwen3 next model (a hybrid attention model). (#24526) · e93f4cc9
  Tao He authored Sep 11, 2025
```
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  e93f4cc9
- [torchao] Support quantization configs using module swap (#21982) · 2048c4e3
  Jerry Zhang authored Sep 10, 2025
```
Signed-off-by: Jerry Zhang <jerryzh168@gmail.com>
```
  2048c4e3
- Remove redundant all gather + split (#23441) · d1336018
  Chenxi Yang authored Sep 10, 2025
```
Co-authored-by: Chenxi Yang <cxyang@meta.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
```
  d1336018
- [Model] New model support for Motif-1-Tiny (#23414) · 9bd831f5
  TaehyunKim authored Sep 11, 2025
```
Signed-off-by: ca1207 <ca1207zzz@gmail.com>
Signed-off-by: TaehyunKim <73943231+ca1207@users.noreply.github.com>
Co-authored-by: WyldeCat <skan1543@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  9bd831f5
- [Doc]: fixing doc typos (#24635) · e2b1f863
  Didier Durand authored Sep 11, 2025
```
Signed-off-by: Didier Durand <durand.didier@gmail.com>
```
  e2b1f863
- [Core] feat: Add --safetensors-load-strategy flag for faster safetensors... · 41329a0f
  shengshiqi-google authored Sep 11, 2025
```
[Core] feat: Add --safetensors-load-strategy flag for faster safetensors loading from Lustre (#24469)
Signed-off-by: Shiqi Sheng <shengshiqi@google.com>
Signed-off-by: shengshiqi-google <160179165+shengshiqi-google@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  41329a0f
- Enable --profile in 'vllm bench throughput' (#24575) · ee0bc5e1
  Tomas Ruiz authored Sep 11, 2025
```
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
```
  ee0bc5e1
- Kimi K2 Fused MoE kernels Optimization configs (#24597) · 3d1393f6
  Saman A. Pour authored Sep 10, 2025
```
Signed-off-by: Saman Keon <samanamp@outlook.com>
```
  3d1393f6
- [Engine][Chore] use local variable and remove output var assignment (#24554) · 8a894084
  Guy Stone authored Sep 11, 2025
```
Signed-off-by: Guy Stone <guys@spotify.com>
```
  8a894084
- [BugFix] Fix pipeline parallel (#24621) · e2d8c27f
  Nick Hill authored Sep 10, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  e2d8c27f
- [Bugfix] Add missing VIT backend dispatch on CPU (#24623) · 29799dda
  Li, Jiang authored Sep 11, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  29799dda
- [Ultravox] Fix Gemma instantiation, support quantization via --hf-overrides (#24131) · f17a6aa4
  Peter Salas authored Sep 10, 2025
```
Signed-off-by: Peter Salas <peter@fixie.ai>
```
  f17a6aa4
- [Bug] [Spec Decode] Fix model_initialization test and mismatch in aux_hidden_layers (#24613) · 6c8deacd
  Wenlong Wang authored Sep 10, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  6c8deacd
- Add @chaunceyjiang to codeowner for reasoning Reasoning and Tool parser (#24406) · 55b823ba
  Chauncey authored Sep 11, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  55b823ba
- [distributed] update known issues (#24624) · 8c5a7472
  youkaichao authored Sep 11, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  8c5a7472
- [Models][Quantization] Add quantization configuration update in Voxtral model (#24122) · 5931b7e5
  Alexandre Marques authored Sep 10, 2025
```
Signed-off-by: Alexandre Marques <almarque@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  5931b7e5
10 Sep, 2025 6 commits

[Misc] Make timeout passable in init_distributed_environment (#24522) · cc99baf1
Jonathan Berkhahn authored Sep 10, 2025
```
Signed-off-by: jberkhahn <jaberkha@us.ibm.com>
```
cc99baf1

[Kernel] Flashinfer MLA (trtllm-gen) decode kernel integration (#21078) · dcb28a33

Hanjie Qiu authored Sep 10, 2025


Signed-off-by: hjjq <hanjieq@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

dcb28a33

[Perf] Warmup FlashInfer attention during startup (#23439) · fba78565

Michael Goin authored Sep 10, 2025


Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni001@gmail.com>

fba78565

[gpt-oss] raise error for flashinfer backend without trtllm (#24482) · b5e383cd
Chen Zhang authored Sep 10, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
b5e383cd

[torch.compile][ROCm][V1] Enable attention output FP8 fusion for V1 attention backends (#19767) · 9a161307

Gregory Shtrasberg authored Sep 10, 2025


Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

9a161307

[v1] Add Whisper model support (encoder-decoder) (#21088) · 37e8182b
Russell Bryant authored Sep 10, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
```
37e8182b