Commits · df845b2b46c3e30f5bd3e3be286285ed148323fc · OpenDAS / vllm_cscc

09 Aug, 2024 1 commit
- [Core] Fix edge case in chunked prefill + block manager v2 (#7380) · baa24025
  Cade Daniel authored Aug 09, 2024
  
  baa24025
08 Aug, 2024 1 commit
- [Bugfix][fast] Fix the get_num_blocks_touched logic (#6849) · 782e53ab
  Zach Zheng authored Aug 08, 2024
  
  782e53ab
06 Aug, 2024 1 commit

[Core] Subclass ModelRunner to support cross-attention & encoder sequences... · fd95e026

afeldman-nm authored Aug 06, 2024


[Core] Subclass ModelRunner to support cross-attention & encoder sequences (towards eventual encoder/decoder model support) (#4942)
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>

fd95e026

19 Jul, 2024 1 commit
- [Misc] Small perf improvements (#6520) · 9ed82e70
  Antoni Baum authored Jul 19, 2024
  
  9ed82e70
02 Jul, 2024 1 commit
- [Core] Optimize block_manager_v2 vs block_manager_v1 (to make V2 default) (#5602) · 3476ed08
  Alexander Matveev authored Jul 01, 2024
  
  3476ed08
15 Jun, 2024 2 commits
- [mypy] Enable type checking for test directory (#5017) · 0e9164b4
  Cyrus Leung authored Jun 15, 2024
  
  0e9164b4
- [Core][Bugfix]: fix prefix caching for blockv2 (#5364) · 1b8a0d71
  leiwen83 authored Jun 15, 2024
```
Signed-off-by: Lei Wen <wenlei03@qiyi.com>
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
```
  1b8a0d71
03 Jun, 2024 1 commit
- [Misc]: Implement CPU/GPU swapping in BlockManagerV2 (#3834) · 10c38e3e
  Kaiyang Chen authored Jun 04, 2024
  
  10c38e3e
29 May, 2024 1 commit
- [Core] Cross-attention KV caching and memory-management (towards eventual... · 4238bc82
  afeldman-nm authored May 29, 2024
```
[Core] Cross-attention KV caching and memory-management (towards eventual encoder/decoder model support) (#4837)
```
  4238bc82
28 May, 2024 1 commit
- [Core] Sliding window for block manager v2 (#4545) · d4f39859
  Michał Moskal authored May 27, 2024
```
Co-authored-by: Ruth Evans <ruthevans@Ruths-MacBook-Pro.local>
```
  d4f39859
24 May, 2024 1 commit
- [Core][Bugfix]: fix prefix caching for blockv2 (#4764) · e64fde4b
  leiwen83 authored May 25, 2024
```
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
```
  e64fde4b
07 May, 2024 1 commit
- [Core][Optimization] change copy-on-write from dict[int, list] to list (#4648) · 469f85c7
  youkaichao authored May 07, 2024
  
  469f85c7
02 May, 2024 2 commits
- [Core] Ignore infeasible swap requests. (#4557) · 0f8a9140
  SangBin Cho authored May 03, 2024
  
  0f8a9140
- [mypy][6/N] Fix all the core subdirectory typing (#4450) · cf8cac8c
  SangBin Cho authored May 02, 2024
```
Co-authored-by: Cade Daniel <edacih@gmail.com>
```
  cf8cac8c
01 May, 2024 1 commit
- [Core] Enable prefix caching with block manager v2 enabled (#4142) · 24750f4c
  leiwen83 authored May 02, 2024
```
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
Co-authored-by: Sage Moore <sagemoore@utexas.edu>
```
  24750f4c
23 Apr, 2024 1 commit
- [Mypy] Part 3 fix typing for nested directories for most of directory (#4161) · 0ae11f78
  SangBin Cho authored Apr 23, 2024
  
  0ae11f78
16 Apr, 2024 1 commit
- [Speculative decoding 6/9] Integrate speculative decoding with LLMEngine (#3894) · e95cd879
  Cade Daniel authored Apr 16, 2024
  
  e95cd879
12 Apr, 2024 1 commit
- [Doc] Add typing hints / mypy types cleanup (#3816) · c2b4a1bc
  Michael Feil authored Apr 11, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  c2b4a1bc
02 Apr, 2024 1 commit
- [Bugfix] Add `__init__.py` files for `vllm/core/block/` and `vllm/spec_decode/` (#3798) · b321d488
  Michael Goin authored Apr 02, 2024
  
  b321d488
01 Apr, 2024 1 commit
- [Speculative decoding 4/9] Lookahead scheduling for speculative decoding (#3250) · 93deb0b3
  Cade Daniel authored Apr 01, 2024
  
  93deb0b3
28 Mar, 2024 1 commit
- [Core][Bugfix]Refactor block manager for better testability (#3492) · 14ccd94c
  Cade Daniel authored Mar 27, 2024
  
  14ccd94c