Commits · bb6888b8b1a03e683ff4ed5f1fb6df5a0582fd6f · OpenDAS / vllm_cscc · GitLab

04 Mar, 2026 1 commit

[Bugfix][CPUOffloadingManager] Prevent eviction of already-stored blocks in... · bb6888b8

Ronen Schaffer authored Mar 04, 2026


[Bugfix][CPUOffloadingManager] Prevent eviction of already-stored blocks in LRU/ARC `prepare_store()` (#35846)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>

bb6888b8

26 Feb, 2026 1 commit
- [BugFix][kv_offload]: Fix kernel block size detection (#35125) · f2ad952f
  Or Ozeri authored Feb 26, 2026
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  f2ad952f
16 Feb, 2026 1 commit
- [Bugfix] Fix ARC touch KeyError for non-ready T1 blocks in kv offload (#34576) · bc56a1d5
  Yiqi Xue authored Feb 16, 2026
```
Signed-off-by: Yiqi Xue <xuey666@gmail.com>
```
  bc56a1d5
27 Jan, 2026 1 commit

[Metrics] [KVConnector] Add Offloading Connector metrics (#27942) · 7227d061

omerpaz95 authored Jan 27, 2026



Added queries and hits metrics for the Offloading Connector.

Also added timing metrics for store and load operations, which take the
average time it takes to load/store, per-token.

The metrics are available from Prometheus and from the StatLogger.
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Co-authored-by: Omer Paz <Omer.Paz@ibm.com>

7227d061

22 Jan, 2026 1 commit
- OffloadingConnector: Support kernel_block_size != block_size (#30692) · 421012b6
  Or Ozeri authored Jan 22, 2026
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  421012b6
21 Jan, 2026 1 commit
- OffloadingConnector: Prevent redundant loads (#29087) · 7013e9ac
  Or Ozeri authored Jan 21, 2026
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  7013e9ac
12 Jan, 2026 1 commit
- OffloadingConnector: Add cpu_bytes_to_use configuration (#24498) · 9cddbdba
  Or Ozeri authored Jan 12, 2026
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  9cddbdba
11 Jan, 2026 1 commit
- [KVConnector] OffloadingConnector: Fix bug in handling of preemptions (#29870) · 4c16ba61
  Or Ozeri authored Jan 11, 2026
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  4c16ba61
10 Jan, 2026 1 commit
- [BugFix] Wait for compute before offloading KV to CPU (#31341) · 2a4dbe24
  Or Ozeri authored Jan 11, 2026
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  2a4dbe24
09 Jan, 2026 1 commit
- [1/N][Attention] Restructure attention: move files (#31916) · 2612ba92
  Matthew Bonanni authored Jan 09, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  2612ba92
14 Dec, 2025 1 commit
- CPU KV Offloading: Use more CUDA streams (#29013) · 174e39ea
  Or Ozeri authored Dec 15, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  174e39ea
27 Nov, 2025 1 commit
- [Attention] Update attention imports (#29540) · fc1d8be3
  Matthew Bonanni authored Nov 27, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  fc1d8be3
26 Nov, 2025 1 commit
- [Attention] Remove imports from `vllm/attention/__init__.py` (#29342) · 430dd4d9
  Matthew Bonanni authored Nov 26, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  430dd4d9
20 Nov, 2025 2 commits
- [KVConnector][Core] Support cross-layer KV blocks (#27743) · 64746471
  Or Ozeri authored Nov 20, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  64746471
- [BugFix] kv_offloading: Fix bug in loading of partial cpu blocks (#28951) · c0c2dd1e
  Or Ozeri authored Nov 20, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  c0c2dd1e
18 Nov, 2025 1 commit
- Replace `torch.cuda.Event` with `torch.Event` for better hardware compatibility (#26985) · 2a2d5d27
  Kunshang Ji authored Nov 19, 2025
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  2a2d5d27
12 Nov, 2025 1 commit

Implement ARC KV cache eviction policy for CPU offloader (#27039) · bac90456

alberto authored Nov 12, 2025


Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
Signed-off-by: alberto <aperdomo@redhat.com>
Co-authored-by: Or Ozeri <or@ozery.com>

bac90456

30 Oct, 2025 1 commit
- [KV offload] Enable CPU KV offload on CUDA alike Platforms (#27770) · 0fe01404
  Zhewen Li authored Oct 30, 2025
```
Signed-off-by: zhewenli <zhewenli@meta.com>
```
  0fe01404
23 Oct, 2025 1 commit
- [Chore] Separate out `vllm.utils.platform_utils.py` (#27374) · ca76486a
  Jonathan Chen authored Oct 23, 2025
```
Signed-off-by: Jonathan <chenleejonathan@gmail.com>
```
  ca76486a
12 Oct, 2025 1 commit
- Update `Optional[x]` -> `x | None` and `Union[x, y]` to `x | y` (#26633) · 8fcaaf6a
  Harry Mellor authored Oct 12, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8fcaaf6a
05 Oct, 2025 1 commit
- Convert formatting to use `ruff` instead of `yapf` + `isort` (#26247) · d6953beb
  Harry Mellor authored Oct 05, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6953beb
24 Sep, 2025 1 commit
- [Docs] Enable `fail_on_warning` for the docs build in CI (#25580) · 8c853050
  Harry Mellor authored Sep 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8c853050
22 Sep, 2025 1 commit
- [KV offload][5/N] Add `CPUOffloadingSpec` (#24251) · 8db29392
  Or Ozeri authored Sep 22, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  8db29392
19 Sep, 2025 3 commits
- [KV offload][4/N] Offloading KV connector (#22595) · c59a0eca
  Or Ozeri authored Sep 19, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  c59a0eca
- [KV offload][3/N] Add worker-side CPU support (#21448) · 7ac67ea5
  Or Ozeri authored Sep 19, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  7ac67ea5
- [KV offload][2/N] Introduce LRU-based CPU offloading management (#20075) · 9d1c50a5
  Or Ozeri authored Sep 19, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  9d1c50a5
18 Sep, 2025 1 commit
- [KV offload][1b/N] rename offloading to kv_offload (#25191) · a53ad626
  Or Ozeri authored Sep 18, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  a53ad626