Commits · 9fac6aa30b669de75d8718164cd99676d3530e7d · OpenDAS / vllm_cscc

"vllm/utils/__init__.py" did not exist on "1744cc99ba9bdefea8f3f798cf51ed650b81a98e"

18 Sep, 2025 13 commits

[KV offload][1b/N] rename offloading to kv_offload (#25191) · a53ad626
Or Ozeri authored Sep 18, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
a53ad626

[Bug] Fix `returned_lse` not Defined issue (#25106) · 75fb112d

Wentao Ye authored Sep 18, 2025


Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

75fb112d

[feat]: Create interface for model-specific M-RoPE (#24194) · 38db529f

Aziz authored Sep 18, 2025

Signed-off-by: AzizCode92 <azizbenothman76@gmail.com>
Signed-off-by: Aziz <azizbenothman76@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

38db529f

[KV offload][1/N] Introduce an offloading component (#19848) · 505805b6
Or Ozeri authored Sep 18, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
505805b6
[ROCm][AITER][Bugfix] Switch AITER to use PIECEWISE_AND_FULL compilation (#25104) · bbdc0f23
Rohan Potdar authored Sep 18, 2025
```
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
```
bbdc0f23

[New Model] Support BertForTokenClassification / Named Entity Recognition (NER) task (#24872) · 5f696c33

wang.yuqi authored Sep 18, 2025


Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

5f696c33

[PERF] Add `conv1d` metadata to GDN attn (#25105) · 072d7e53
Vadim Gimpelson authored Sep 18, 2025
```
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
```
072d7e53

[Structured Output][Refactor] Move `apply_grammar_bitmask()` method from... · 470484a4

Shanshan Shen authored Sep 18, 2025


[Structured Output][Refactor] Move `apply_grammar_bitmask()` method from `ModelRunner` to structured output utils (#21999)
Signed-off-by: shen-shanshan <467638484@qq.com>

470484a4

[Model] Improve Pooling Model (#25149) · 37970105
Jee Jee Li authored Sep 18, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
37970105

[Chore] Cleanup guided namespace, move to structured outputs config (#22772) · 29283e89

Aaron Pham authored Sep 18, 2025

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

29283e89

[Doc] Fix cross-reference warnings (#25058) · 05b044e6

Punitvara authored Sep 18, 2025

Signed-off-by: Punit Vara <punitvara@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

05b044e6

[Spec Decode] Efficient padded speculation (#24539) · b7433ca1
Benjamin Chislett authored Sep 18, 2025
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
b7433ca1
[XPU] Whisper model support on XPU Platform (#25123) · 3bc18127
Chaojun Zhang authored Sep 18, 2025
```
Signed-off-by: chzhang <chaojun.zhang@intel.com>
```
3bc18127

17 Sep, 2025 12 commits
- [Bugfix] Refactor Flashinfer TRTLLM attention kernel selection logic (#24600) · e67a79db
  elvischenv authored Sep 18, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  e67a79db
- Aiter mha fp8 fix (#24991) · 1a456c7c
  Douglas Lehr authored Sep 17, 2025
```
Signed-off-by: Doug Lehr <douglehr@amd.com>
Co-authored-by: Doug Lehr <douglehr@amd.com>
```
  1a456c7c
- [Bugfix] Update import path for bc_linter_include (#24766) · 88313154
  Mohammad Miadh Angkad authored Sep 18, 2025
```
Signed-off-by: Mohammad Miadh Angkad <mangkad.bsdsba2027@aim.edu>
```
  88313154
- [V1] Logits processor docs (#22919) · 7ae98875
  afeldman-nm authored Sep 17, 2025
```
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Signed-off-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Joseph Marinier <Joseph.Marinier@gmail.com>
```
  7ae98875
- Remove old cutlass mla (#23961) · 8f3616f4
  Matthew Bonanni authored Sep 17, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  8f3616f4
- [Bugfix][Qwen3-Next] fixes the varlen issue in qwen3-next's MTP implementation. (#24957) · dd6a910a
  Tao He authored Sep 17, 2025
```
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
```
  dd6a910a
- [Bugfix] Fix Stream usage in CPU model runner and OneDNN kernel check (#25046) · 9fccd04e
  Li, Jiang authored Sep 17, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  9fccd04e
- [DP] Create placement groups by ray_device_key (#25026) · bb58dc8c
  Xinyu Chen authored Sep 17, 2025
```
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
```
  bb58dc8c
- [Core] Remove tokenizer group in vLLM (#24078) · 6c47f6bf
  Zhuohan Li authored Sep 17, 2025
```
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
```
  6c47f6bf
- [XPU] Fix xpu model runner call torch.cuda APIs (#25011) · dd39baf7
  Kunshang Ji authored Sep 17, 2025
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  dd39baf7
- [Core] Get num_encoder_tokens from scheduler config (#24989) · 58d4c705
  Russell Bryant authored Sep 16, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  58d4c705
- [Core] Use `CpuGpuBuffer` for block table tensors (#24795) · eeb135eb
  Nick Hill authored Sep 16, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  eeb135eb
16 Sep, 2025 7 commits

fp8 kv cache support fix for torch.compile (#22758) · 3053a22b

Aleksandr Malyshev authored Sep 16, 2025

Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>

3053a22b

[Bugfix] Fix unable to run encoder model when disable_hybrid_kv_cache_manager is true (#24571) · faa7a5da
lianyibo authored Sep 17, 2025
```
Signed-off-by: lianyibo <lianyibo1@kunlunit.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
```
faa7a5da

[Core/DBO][1/N] Add Dual-Batch Overlap mechanism to VLLM (#23693) · 56793995

Sage Moore authored Sep 16, 2025


Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

56793995

[Bugfix] Fix sequence parallelism bug when enable pipeline parallelism (#24021) · 17871983
cascade authored Sep 15, 2025
```
Signed-off-by: cascade812 <cascade812@outlook.com>
```
17871983
Remove V0 Encoder-Decoder Support (#24907) · 759ef49b
Woosuk Kwon authored Sep 15, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
759ef49b
[Metrics] Hide deprecated metrics with gpu_ prefix (#24245) · 2942970d
Mark McLoughlin authored Sep 16, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
2942970d
[Bug] Fix `is_flashmla_supported` Check Error (#24774) · b42566f4
Wentao Ye authored Sep 15, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
b42566f4

15 Sep, 2025 6 commits
- [Performance] Remove redundant clone() calls in cutlass_mla (#24891) · aae725af
  Alexander Matveev authored Sep 15, 2025
  
  aae725af
- [gpt-oss] Add IncompleteDetails to ResponsesRepsonse (#24561) · 25aba2b6
  Andrew Xia authored Sep 15, 2025
```
Signed-off-by: Andrew Xia <axia@meta.com>
```
  25aba2b6
- [XPU] Set consistent default KV cache layout (#24745) · 2e41f5ab
  Nicolò Lucchesi authored Sep 15, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  2e41f5ab
- [P/D]`kv_output_aggregator` support P TP > D TP (#23917) · 8de261b0
  Chao Lei authored Sep 15, 2025
```
Signed-off-by: LCAIZJ <leichao139636@163.com>
Co-authored-by: leichao.lc <leichao.lc@antgroup.com>
```
  8de261b0
- [Misc] rename interval to max_recent_requests (#24229) · 59e17dd4
  Ning Xie authored Sep 15, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  59e17dd4
- [kv cache] update num_free_blocks in the end (#24228) · 3f331398
  Ning Xie authored Sep 15, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  3f331398
14 Sep, 2025 2 commits
- [Hybrid Allocator] Support Pipeline Parallel (#23974) · 8e5cdcda
  Chen Zhang authored Sep 14, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  8e5cdcda
- [Spec Decoding]Support Spec Decoding Metrics in DP Mode (#24049) · 90f3f7d7
  wuhang authored Sep 15, 2025
```
Signed-off-by: wuhang <wuhang6@huawei.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
```
  90f3f7d7