Commits · cc1dc7ed6d2d901df46847724ad90147872a9d79 · OpenDAS / vllm_cscc

23 Sep, 2025 6 commits

[Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) · cc1dc7ed

Lucas Wilkinson authored Sep 23, 2025


Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

cc1dc7ed

[Docs] NixlConnector quickstart guide (#24249) · da5e7e43

Peter Pan authored Sep 23, 2025


Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
Signed-off-by: Peter Pan <peter.pan@daocloud.io>
Signed-off-by: Nicolò Lucchesi<nicolo.lucchesi@gmail.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>

da5e7e43

[P/D] Support NIXL connector to disconnect during a clean shutdown (#24423) · f05a4f0e
Chauncey authored Sep 23, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
```
f05a4f0e
[NIXL][OOT platform] support nixl_connector with oot platform and other nixl_backend (#25121) · 5774b0a1
Chendi.Xue authored Sep 22, 2025
```
Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>
```
5774b0a1
[V0 deprecation] Remove platform v1 controling interface (#25410) · 6fa78d8f
Isotr0py authored Sep 23, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
6fa78d8f
[Bugfix] Fix missing `clear_connector_metadata` (#25397) · 09019703
Nicolò Lucchesi authored Sep 23, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
09019703

22 Sep, 2025 3 commits
- [KV offload][5/N] Add `CPUOffloadingSpec` (#24251) · 8db29392
  Or Ozeri authored Sep 22, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  8db29392
- [CLI env var] Add VLLM_FLASH_ATTN_MAX_NUM_SPLITS_FOR_CUDA_GRAPH in env variables (#25274) · cfbee3d0
  Daisy-Ma-coder authored Sep 22, 2025
```
Signed-off-by: qqma <qqma@amazon.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: qqma <qqma@amazon.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  cfbee3d0
- [TPU] Deprecate `xm.mark_step` in favor of ``torch_xla.sync` (#25254) · 4cf71cc8
  Nicolò Lucchesi authored Sep 22, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
```
  4cf71cc8
21 Sep, 2025 3 commits
- [V0 Deprecation] Remove async_output_proc, preemption mode, delay factor (#25334) · 0ff8ebb2
  Woosuk Kwon authored Sep 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  0ff8ebb2
- [V0 Deprecation] Remove V0 Sequence class & Sampler (#25332) · 26e673fe
  Woosuk Kwon authored Sep 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
  26e673fe
- [V1] Add sliding window support to Flex Attention backend (#24089) · cf56cf78
  Isotr0py authored Sep 21, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  cf56cf78
20 Sep, 2025 2 commits
- [Hybrid Allocator] Support full attention with different hidden size (#25101) · 9607d5eb
  Chen Zhang authored Sep 19, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  9607d5eb
- [Misc] Support more collective_rpc return types (#25294) · 535d8005
  Nick Hill authored Sep 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  535d8005
19 Sep, 2025 7 commits
- [KV offload][4/N] Offloading KV connector (#22595) · c59a0eca
  Or Ozeri authored Sep 19, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  c59a0eca
- [Core][Prefix Hash] Fix prefix hash metrics sliding window maintainance (#24990) · 2506ce51
  Jialin Ouyang authored Sep 19, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  2506ce51
- [Frontend] Pass API server count to each process (#23717) · 6c117cff
  Cyrus Leung authored Sep 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6c117cff
- [KV offload][3/N] Add worker-side CPU support (#21448) · 7ac67ea5
  Or Ozeri authored Sep 19, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  7ac67ea5
- Move `ModelConfig` from `config/__init__.py` to `config/model.py` (#25252) · aed16879
  Harry Mellor authored Sep 19, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  aed16879
- [P/D][Nixl] Introduce `KVTransferMetrics` and aggregation strategy (#22188) · a3d087ad
  Nicolò Lucchesi authored Sep 19, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  a3d087ad
- [KV offload][2/N] Introduce LRU-based CPU offloading management (#20075) · 9d1c50a5
  Or Ozeri authored Sep 19, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  9d1c50a5
18 Sep, 2025 6 commits
- [KV offload][1b/N] rename offloading to kv_offload (#25191) · a53ad626
  Or Ozeri authored Sep 18, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  a53ad626
- [V0 Deprecation] Remove AsyncLLMEngine (#25025) · e19bce40
  Woosuk Kwon authored Sep 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e19bce40
- [KV offload][1/N] Introduce an offloading component (#19848) · 505805b6
  Or Ozeri authored Sep 18, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  505805b6
- [Chore] Cleanup guided namespace, move to structured outputs config (#22772) · 29283e89
  Aaron Pham authored Sep 18, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  29283e89
- [Spec Decode] Efficient padded speculation (#24539) · b7433ca1
  Benjamin Chislett authored Sep 18, 2025
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
  b7433ca1
- [V0 Deprecation] Remove V0 Tracing & Metrics tests (#25115) · 2fc24e94
  Woosuk Kwon authored Sep 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2fc24e94
17 Sep, 2025 4 commits
- [V1] Logits processor docs (#22919) · 7ae98875
  afeldman-nm authored Sep 17, 2025
```
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Signed-off-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Joseph Marinier <Joseph.Marinier@gmail.com>
```
  7ae98875
- [CI] Revert back prepare_prompts and check_answers (#25087) · eb68c2dc
  Woosuk Kwon authored Sep 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  eb68c2dc
- [Core] Remove tokenizer group in vLLM (#24078) · 6c47f6bf
  Zhuohan Li authored Sep 17, 2025
```
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
```
  6c47f6bf
- [Core] Use `CpuGpuBuffer` for block table tensors (#24795) · eeb135eb
  Nick Hill authored Sep 16, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  eeb135eb
16 Sep, 2025 2 commits

[Core/DBO][1/N] Add Dual-Batch Overlap mechanism to VLLM (#23693) · 56793995

Sage Moore authored Sep 16, 2025


Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

56793995

Remove V0 Encoder-Decoder Support (#24907) · 759ef49b
Woosuk Kwon authored Sep 15, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
759ef49b

15 Sep, 2025 4 commits
- [Tests] fix initialization of kv hash in tests (#24273) · 45bfa49c
  Mickaël Seznec authored Sep 15, 2025
```
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
```
  45bfa49c
- Move `MultiModalConfig` from `config/__init__.py` to `config/multimodal.py` (#24659) · c4afdb69
  Harry Mellor authored Sep 15, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  c4afdb69
- [UT] enhance free kv cache block queue popleft_n (#24220) · bc0f6059
  Ning Xie authored Sep 15, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  bc0f6059
- [kv cache] update num_free_blocks in the end (#24228) · 3f331398
  Ning Xie authored Sep 15, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  3f331398
14 Sep, 2025 1 commit
- [Hybrid Allocator] Support Pipeline Parallel (#23974) · 8e5cdcda
  Chen Zhang authored Sep 14, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  8e5cdcda
13 Sep, 2025 1 commit
- [CI][Spec Decode] Adjust threshold for flaky ngram spec decoding test again (#24771) · cfa3234a
  Wenlong Wang authored Sep 13, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  cfa3234a
12 Sep, 2025 1 commit

[Core] Support async scheduling with uniproc executor (#24219) · 4fdd6f5c

Nick Hill authored Sep 12, 2025


Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

4fdd6f5c