Commits · 73001445fbfc42d386d68066519738dfffa62df3 · OpenDAS / vllm_cscc

01 Jan, 2025 1 commit
- [V1] Implement Cascade Attention (#11635) · 73001445
  Woosuk Kwon authored Jan 01, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  73001445
31 Dec, 2024 2 commits
- [V1] Simpify vision block hash for prefix caching by removing offset from hash (#11646) · 8c3230d8
  Chen Zhang authored Dec 31, 2024
  
  8c3230d8
- [Bugfix] Move the _touch(computed_blocks) call in the allocate_slots method to... · 2c571880
  sakunkun authored Dec 31, 2024
```
[Bugfix] Move the _touch(computed_blocks) call in the allocate_slots method to after the check for allocating new blocks. (#11565)
```
  2c571880
19 Dec, 2024 1 commit
- [V1] Simplify prefix caching logic by removing `num_evictable_computed_blocks` (#11310) · c6b0a7d3
  Chen Zhang authored Dec 18, 2024
  
  c6b0a7d3
18 Dec, 2024 2 commits
- [V1] VLM - enable processor cache by default (#11305) · fdea8ec1
  Alexander Matveev authored Dec 18, 2024
```
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
```
  fdea8ec1
- [V1] Prefix caching for vision language models (#11187) · bf8717eb
  Cody Yu authored Dec 17, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  bf8717eb
17 Dec, 2024 1 commit
- [V1][VLM] Proper memory profiling for image language models (#11210) · 59c9b6eb
  Roger Wang authored Dec 16, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: ywang96 <ywang@example.com>
```
  59c9b6eb
14 Dec, 2024 2 commits
- Enable mypy checking on V1 code (#11105) · 6d917d0e
  Mark McLoughlin authored Dec 14, 2024
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  6d917d0e
- [Bugfix][V1] Re-compute an entire block when fully cache hit (#11186) · 9855aea2
  Cody Yu authored Dec 13, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  9855aea2
13 Dec, 2024 2 commits
- [Bugfix][V1] Fix 'NoneType' object has no attribute 'hash_value' (#11157) · 34f1a806
  Cody Yu authored Dec 12, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  34f1a806
- [Misc][V1] Fix type in v1 prefix caching (#11151) · 78ed8f57
  Cody Yu authored Dec 12, 2024
  
  78ed8f57
10 Dec, 2024 1 commit
- [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) · 28b3a1c7
  Tyler Michael Smith authored Dec 10, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  28b3a1c7
08 Dec, 2024 1 commit
- [V1] Initial support of multimodal models for V1 re-arch (#10699) · a11f3265
  Roger Wang authored Dec 08, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  a11f3265
28 Nov, 2024 1 commit
- [V1] Do not allocate beyond the max_model_len (#10730) · a79b1224
  Woosuk Kwon authored Nov 28, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  a79b1224
22 Nov, 2024 1 commit

[v1] Refactor KVCacheManager for more hash input than token ids (#10507) · 97814fbf

Ricky Xu authored Nov 22, 2024


Signed-off-by: rickyx <rickyx@anyscale.com>
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>

97814fbf

13 Nov, 2024 1 commit

[V1] Support VLMs with fine-grained scheduling (#9871) · bbd3e869

Woosuk Kwon authored Nov 12, 2024


Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <ywang@roblox.com>

bbd3e869

11 Nov, 2024 1 commit

[V1] `AsyncLLM` Implementation (#9826) · 6ace6fba

Robert Shaw authored Nov 11, 2024


Signed-off-by: Nick Hill <nickhill@us.ibm.com>
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

6ace6fba

08 Nov, 2024 2 commits
- [V1] Prefix caching (take 2) (#9972) · 201fc077
  Cody Yu authored Nov 07, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  201fc077
- [V1] Add all_token_ids attribute to Request (#10135) · 42b4f46b
  Woosuk Kwon authored Nov 07, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  42b4f46b
22 Oct, 2024 1 commit
- [V1] Implement vLLM V1 [1/N] (#9289) · 6c5af09b
  Woosuk Kwon authored Oct 22, 2024
  
  6c5af09b