Commits · 9474e89ba4ecae253b585eb6b3e1d85f4e108f01 · OpenDAS / vllm_cscc

20 Mar, 2024 1 commit

[PREFIX CACHING FOLLOW UP] A bunch of fixes to block allocator performance... · 9474e89b

ElizaWszola authored Mar 20, 2024


[PREFIX CACHING FOLLOW UP] A bunch of fixes to block allocator performance when automatic prefix caching is disabled (#3357)
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

9474e89b

13 Mar, 2024 1 commit
- Fixes #1556 double free (#3347) · 49a3c866
  Breno Faria authored Mar 13, 2024
  
  49a3c866
11 Mar, 2024 1 commit
- Re-enable the 80 char line width limit (#3305) · 2f8844ba
  Zhuohan Li authored Mar 10, 2024
  
  2f8844ba
08 Mar, 2024 1 commit
- Fix auto prefix bug (#3239) · b35cc934
  ElizaWszola authored Mar 08, 2024
  
  b35cc934
03 Mar, 2024 1 commit
- [FIX] Fix styles in automatic prefix caching & add a automatic prefix caching benchmark (#3158) · 996d095c
  Zhuohan Li authored Mar 03, 2024
  
  996d095c
02 Mar, 2024 1 commit

Add Automatic Prefix Caching (#2762) · ce4f5a29

Sage Moore authored Mar 02, 2024


Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

ce4f5a29

22 Feb, 2024 1 commit
- chore(vllm): codespell for spell checking (#2820) · 93dc5a28
  Massimiliano Pronesti authored Feb 22, 2024
  
  93dc5a28
18 Jan, 2024 1 commit

[Experimental] Prefix Caching Support (#1669) · d10f8e1d

shiyi.c_98 authored Jan 17, 2024


Co-authored-by: DouHappy <2278958187@qq.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

d10f8e1d

21 Dec, 2023 1 commit
- [BugFix] Fix recovery logic for sequence group (#2186) · a1b9cb2a
  Woosuk Kwon authored Dec 20, 2023
  
  a1b9cb2a
29 Nov, 2023 2 commits
- [FIX] Fix formatting error · 1cb4ad8d
  Zhuohan Li authored Nov 29, 2023
  
  1cb4ad8d
- Use the type BlockTable (#1791) · 6ed068a7
  explainerauthors authored Nov 28, 2023
  
  6ed068a7
21 Nov, 2023 1 commit
- Fix hanging in the scheduler caused by long prompts (#1534) · 3d4ceb29
  陈序 authored Nov 21, 2023
  
  3d4ceb29
28 Sep, 2023 1 commit
- [Mistral] Mistral-7B-v0.1 support (#1196) · bb1ba58f
  Chris Bamford authored Sep 28, 2023
```
Co-authored-by: timlacroix <t@mistral.ai>
```
  bb1ba58f
05 Sep, 2023 1 commit
- Align vLLM's beam search implementation with HF generate (#857) · 002800f0
  Zhuohan Li authored Sep 04, 2023
  
  002800f0
03 Jul, 2023 1 commit
- [Quality] Add code formatter and linter (#326) · d6fa1be3
  Zhuohan Li authored Jul 03, 2023
  
  d6fa1be3
17 Jun, 2023 1 commit
- Change the name to vLLM (#150) · 0b98ba15
  Woosuk Kwon authored Jun 17, 2023
  
  0b98ba15
05 Jun, 2023 1 commit
- Fix various issues of async servers (#135) · 1a956e13
  Zhuohan Li authored Jun 05, 2023
  
  1a956e13
24 May, 2023 1 commit
- OpenAI Compatible Frontend (#116) · 057daef7
  Zhuohan Li authored May 23, 2023
  
  057daef7
21 May, 2023 1 commit
- Implement stop strings and best_of (#114) · f746ced0
  Woosuk Kwon authored May 21, 2023
  
  f746ced0
15 May, 2023 1 commit
- Add docstrings to some modules and classes (#100) · b322fd16
  Woosuk Kwon authored May 14, 2023
  
  b322fd16
10 May, 2023 1 commit
- Rename variables and methods (#91) · 8d66a7b6
  Woosuk Kwon authored May 10, 2023
  
  8d66a7b6
09 May, 2023 1 commit
- Refactor system architecture (#82) · 7c041ab5
  Woosuk Kwon authored May 09, 2023
  
  7c041ab5
15 Apr, 2023 1 commit
- Support various block sizes & Change default block size to 16 (#38) · 0f4b3219
  Woosuk Kwon authored Apr 15, 2023
  
  0f4b3219
12 Apr, 2023 1 commit
- Collect system stats in scheduler & Add scripts for experiments (#30) · 84eee24e
  Woosuk Kwon authored Apr 12, 2023
  
  84eee24e
10 Apr, 2023 1 commit
- Support block size 32 (#35) · b9926f7f
  Woosuk Kwon authored Apr 09, 2023
  
  b9926f7f
30 Mar, 2023 1 commit
- Implement preemption via recomputation & Refactor scheduling logic (#12) · 7a7929ab
  Woosuk Kwon authored Mar 30, 2023
  
  7a7929ab
29 Mar, 2023 1 commit
- Add cache watermark to avoid frequent cache eviction (#11) · 64e0e383
  Woosuk Kwon authored Mar 29, 2023
  
  64e0e383
26 Mar, 2023 1 commit
- Minor · d359cda5
  Woosuk Kwon authored Mar 26, 2023
  
  d359cda5
01 Mar, 2023 1 commit
- Implement `single_query_cached_kv_attention` kernel (#3) · 0deacbce
  Woosuk Kwon authored Mar 01, 2023
  
  0deacbce
23 Feb, 2023 1 commit
- Add get_block_table · 4f6f4967
  Woosuk Kwon authored Feb 23, 2023
  
  4f6f4967
14 Feb, 2023 7 commits
- Fix a ref count bug in swapping · 2729087e
  Woosuk Kwon authored Feb 14, 2023
  
  2729087e
- Fix double free · c128c2ed
  Woosuk Kwon authored Feb 14, 2023
  
  c128c2ed
- Minor · c80ed212
  Woosuk Kwon authored Feb 14, 2023
  
  c80ed212
- Minor · 263e91b3
  Woosuk Kwon authored Feb 14, 2023
  
  263e91b3
- Fix memory leak in swapping · be1e2163
  Woosuk Kwon authored Feb 14, 2023
  
  be1e2163
- Fix a bug in swap_in and swap_out · 1f739f9b
  Woosuk Kwon authored Feb 14, 2023
  
  1f739f9b
- Fix can_swap_in · b1644f76
  Woosuk Kwon authored Feb 14, 2023
  
  b1644f76
13 Feb, 2023 1 commit
- Fix can_swap_in · eb52db1b
  Woosuk Kwon authored Feb 13, 2023
  
  eb52db1b
12 Feb, 2023 2 commits
- SERVING -> RUNNING · a2a9869c
  Woosuk Kwon authored Feb 12, 2023
  
  a2a9869c
- BlockAllocator -> BlockManager · 46958cf9
  Woosuk Kwon authored Feb 12, 2023
  
  46958cf9