Commits · ee7a66dd9a5ead46f062502af33766f45076f05d · OpenDAS / vllm_cscc

"vllm/vscode:/vscode.git/clone" did not exist on "d4d5291cc216ccc7ce824c03ab25141026b9a394"

19 Sep, 2025 12 commits
- allow disable flashinfer prefill (#25276) · ee7a66dd
  Lucia Fang authored Sep 19, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
```
  ee7a66dd
- [BugFix] Make FlashInferMetadataBuilder non-blocking (#25040) · b1a63d1b
  nvjullin authored Sep 20, 2025
```
Signed-off-by: Julien Lin <jullin@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  b1a63d1b
- [KV offload][4/N] Offloading KV connector (#22595) · c59a0eca
  Or Ozeri authored Sep 19, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  c59a0eca
- [bugfix] fix structured outputs key missing issue from #24929 (#25195) · b716ab93
  Lucia Fang authored Sep 19, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
```
  b716ab93
- [Core][Prefix Hash] Fix prefix hash metrics sliding window maintainance (#24990) · 2506ce51
  Jialin Ouyang authored Sep 19, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  2506ce51
- [Core] Modify the initialization parameters of the lora manager (#25249) · 28219864
  Jee Jee Li authored Sep 20, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  28219864
- [Frontend] Pass API server count to each process (#23717) · 6c117cff
  Cyrus Leung authored Sep 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6c117cff
- [KV offload][3/N] Add worker-side CPU support (#21448) · 7ac67ea5
  Or Ozeri authored Sep 19, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  7ac67ea5
- Move `ModelConfig` from `config/__init__.py` to `config/model.py` (#25252) · aed16879
  Harry Mellor authored Sep 19, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  aed16879
- [P/D][Nixl] Introduce `KVTransferMetrics` and aggregation strategy (#22188) · a3d087ad
  Nicolò Lucchesi authored Sep 19, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  a3d087ad
- [KV offload][2/N] Introduce LRU-based CPU offloading management (#20075) · 9d1c50a5
  Or Ozeri authored Sep 19, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  9d1c50a5
- [CORE] Prompt Embeddings Support for v1 Engine (#24278) · 9a4600e4
  Andrew Sansom authored Sep 18, 2025
```
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: Andrew Sansom <qthequartermasterman@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  9a4600e4
18 Sep, 2025 13 commits

[KV offload][1b/N] rename offloading to kv_offload (#25191) · a53ad626
Or Ozeri authored Sep 18, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
a53ad626

[Bug] Fix `returned_lse` not Defined issue (#25106) · 75fb112d

Wentao Ye authored Sep 18, 2025


Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

75fb112d

[feat]: Create interface for model-specific M-RoPE (#24194) · 38db529f

Aziz authored Sep 18, 2025

Signed-off-by: AzizCode92 <azizbenothman76@gmail.com>
Signed-off-by: Aziz <azizbenothman76@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

38db529f

[KV offload][1/N] Introduce an offloading component (#19848) · 505805b6
Or Ozeri authored Sep 18, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
505805b6
[ROCm][AITER][Bugfix] Switch AITER to use PIECEWISE_AND_FULL compilation (#25104) · bbdc0f23
Rohan Potdar authored Sep 18, 2025
```
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
```
bbdc0f23

[New Model] Support BertForTokenClassification / Named Entity Recognition (NER) task (#24872) · 5f696c33

wang.yuqi authored Sep 18, 2025


Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

5f696c33

[PERF] Add `conv1d` metadata to GDN attn (#25105) · 072d7e53
Vadim Gimpelson authored Sep 18, 2025
```
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
```
072d7e53

[Structured Output][Refactor] Move `apply_grammar_bitmask()` method from... · 470484a4

Shanshan Shen authored Sep 18, 2025


[Structured Output][Refactor] Move `apply_grammar_bitmask()` method from `ModelRunner` to structured output utils (#21999)
Signed-off-by: shen-shanshan <467638484@qq.com>

470484a4

[Model] Improve Pooling Model (#25149) · 37970105
Jee Jee Li authored Sep 18, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
37970105

[Chore] Cleanup guided namespace, move to structured outputs config (#22772) · 29283e89

Aaron Pham authored Sep 18, 2025

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

29283e89

[Doc] Fix cross-reference warnings (#25058) · 05b044e6

Punitvara authored Sep 18, 2025

Signed-off-by: Punit Vara <punitvara@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

05b044e6

[Spec Decode] Efficient padded speculation (#24539) · b7433ca1
Benjamin Chislett authored Sep 18, 2025
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
b7433ca1
[XPU] Whisper model support on XPU Platform (#25123) · 3bc18127
Chaojun Zhang authored Sep 18, 2025
```
Signed-off-by: chzhang <chaojun.zhang@intel.com>
```
3bc18127

17 Sep, 2025 12 commits
- [Bugfix] Refactor Flashinfer TRTLLM attention kernel selection logic (#24600) · e67a79db
  elvischenv authored Sep 18, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  e67a79db
- Aiter mha fp8 fix (#24991) · 1a456c7c
  Douglas Lehr authored Sep 17, 2025
```
Signed-off-by: Doug Lehr <douglehr@amd.com>
Co-authored-by: Doug Lehr <douglehr@amd.com>
```
  1a456c7c
- [Bugfix] Update import path for bc_linter_include (#24766) · 88313154
  Mohammad Miadh Angkad authored Sep 18, 2025
```
Signed-off-by: Mohammad Miadh Angkad <mangkad.bsdsba2027@aim.edu>
```
  88313154
- [V1] Logits processor docs (#22919) · 7ae98875
  afeldman-nm authored Sep 17, 2025
```
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Signed-off-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Joseph Marinier <Joseph.Marinier@gmail.com>
```
  7ae98875
- Remove old cutlass mla (#23961) · 8f3616f4
  Matthew Bonanni authored Sep 17, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  8f3616f4
- [Bugfix][Qwen3-Next] fixes the varlen issue in qwen3-next's MTP implementation. (#24957) · dd6a910a
  Tao He authored Sep 17, 2025
```
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
```
  dd6a910a
- [Bugfix] Fix Stream usage in CPU model runner and OneDNN kernel check (#25046) · 9fccd04e
  Li, Jiang authored Sep 17, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  9fccd04e
- [DP] Create placement groups by ray_device_key (#25026) · bb58dc8c
  Xinyu Chen authored Sep 17, 2025
```
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
```
  bb58dc8c
- [Core] Remove tokenizer group in vLLM (#24078) · 6c47f6bf
  Zhuohan Li authored Sep 17, 2025
```
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
```
  6c47f6bf
- [XPU] Fix xpu model runner call torch.cuda APIs (#25011) · dd39baf7
  Kunshang Ji authored Sep 17, 2025
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  dd39baf7
- [Core] Get num_encoder_tokens from scheduler config (#24989) · 58d4c705
  Russell Bryant authored Sep 16, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  58d4c705
- [Core] Use `CpuGpuBuffer` for block table tensors (#24795) · eeb135eb
  Nick Hill authored Sep 16, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  eeb135eb
16 Sep, 2025 3 commits

fp8 kv cache support fix for torch.compile (#22758) · 3053a22b

Aleksandr Malyshev authored Sep 16, 2025

Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>

3053a22b

[Bugfix] Fix unable to run encoder model when disable_hybrid_kv_cache_manager is true (#24571) · faa7a5da
lianyibo authored Sep 17, 2025
```
Signed-off-by: lianyibo <lianyibo1@kunlunit.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
```
faa7a5da

[Core/DBO][1/N] Add Dual-Batch Overlap mechanism to VLLM (#23693) · 56793995

Sage Moore authored Sep 16, 2025


Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

56793995