Commits · 8917782af6a5892a0afb697badbe761dc8558619 · norm / vllm

09 May, 2023 1 commit
- Add a system logger (#85) · 8917782a
  Woosuk Kwon authored May 08, 2023
  
  8917782a
05 May, 2023 1 commit
- Replace FlashAttention with xformers (#70) · c9d5b6d4
  Woosuk Kwon authored May 05, 2023
  
  c9d5b6d4
04 May, 2023 1 commit
- Use dtype from model config & Add Dolly V2 (#63) · 189ae231
  Woosuk Kwon authored May 04, 2023
  
  189ae231
03 May, 2023 2 commits
- Support bfloat16 data type (#54) · e070829a
  Woosuk Kwon authored May 03, 2023
  
  e070829a
- New weight loader without np copy (#52) · 27f1410d
  Zhuohan Li authored May 03, 2023
  
  27f1410d
30 Apr, 2023 1 commit
- Add an option to launch cacheflow without ray (#51) · 4858f3bb
  Zhuohan Li authored Apr 30, 2023
  
  4858f3bb
15 Apr, 2023 1 commit
- Support various block sizes & Change default block size to 16 (#38) · 0f4b3219
  Woosuk Kwon authored Apr 15, 2023
  
  0f4b3219
12 Apr, 2023 1 commit
- Collect system stats in scheduler & Add scripts for experiments (#30) · 84eee24e
  Woosuk Kwon authored Apr 12, 2023
  
  84eee24e
10 Apr, 2023 1 commit
- Support block size 32 (#35) · b9926f7f
  Woosuk Kwon authored Apr 09, 2023
  
  b9926f7f
09 Apr, 2023 1 commit
- Add an option to use dummy model weights (#33) · ee88a7e5
  Woosuk Kwon authored Apr 08, 2023
  
  ee88a7e5
05 Apr, 2023 1 commit
- Add CUDA graph-based all reduce launcher (#26) · 12659a0b
  Woosuk Kwon authored Apr 05, 2023
  
  12659a0b
30 Mar, 2023 2 commits
- Implement preemption via recomputation & Refactor scheduling logic (#12) · 7a7929ab
  Woosuk Kwon authored Mar 30, 2023
  
  7a7929ab
- Implement LLaMA (#9) · 80a2f812
  Woosuk Kwon authored Mar 29, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  80a2f812
29 Mar, 2023 2 commits
- Add cache watermark to avoid frequent cache eviction (#11) · 64e0e383
  Woosuk Kwon authored Mar 29, 2023
  
  64e0e383
- FastAPI-based working frontend (#10) · 721fa3df
  Zhuohan Li authored Mar 29, 2023
  
  721fa3df
26 Mar, 2023 1 commit
- Minor · d359cda5
  Woosuk Kwon authored Mar 26, 2023
  
  d359cda5
13 Mar, 2023 1 commit
- Add miscellaneous updates (#8) · cfae35b8
  Woosuk Kwon authored Mar 13, 2023
  
  cfae35b8
12 Mar, 2023 1 commit
- Add memory analyzer & utomatically configure KV cache size (#6) · e9d3f2ff
  Woosuk Kwon authored Mar 11, 2023
  
  e9d3f2ff
10 Mar, 2023 1 commit
- Support beam search & parallel generation (#7) · 1a7eb7da
  Woosuk Kwon authored Mar 10, 2023
  
  1a7eb7da
01 Mar, 2023 1 commit
- Implement `single_query_cached_kv_attention` kernel (#3) · 0deacbce
  Woosuk Kwon authored Mar 01, 2023
  
  0deacbce
24 Feb, 2023 5 commits
- [Minor] Fix printing format · 6aef2278
  Woosuk Kwon authored Feb 24, 2023
  
  6aef2278
- Add Frontend · 1132fae0
  Woosuk Kwon authored Feb 24, 2023
  
  1132fae0
- Merge pre_step and step · ef6098ec
  Woosuk Kwon authored Feb 24, 2023
  
  ef6098ec
- Reduce the number of states in scheduler · 53f70e73
  Woosuk Kwon authored Feb 24, 2023
  
  53f70e73
- [WIP] Add server script · afdbe5d3
  Woosuk Kwon authored Feb 24, 2023
  
  afdbe5d3
23 Feb, 2023 3 commits
- Minor · fdd0f2f4
  Woosuk Kwon authored Feb 23, 2023
  
  fdd0f2f4
- Add get_block_table · 4f6f4967
  Woosuk Kwon authored Feb 23, 2023
  
  4f6f4967
- Implement scheduler.step & Add a threshold for batch size · 331fa0b0
  Woosuk Kwon authored Feb 23, 2023
  
  331fa0b0
14 Feb, 2023 8 commits
- Fix a ref count bug in swapping · 2729087e
  Woosuk Kwon authored Feb 14, 2023
  
  2729087e
- Fix double free · c128c2ed
  Woosuk Kwon authored Feb 14, 2023
  
  c128c2ed
- Minor · c80ed212
  Woosuk Kwon authored Feb 14, 2023
  
  c80ed212
- Minor · 263e91b3
  Woosuk Kwon authored Feb 14, 2023
  
  263e91b3
- Fix memory leak in swapping · be1e2163
  Woosuk Kwon authored Feb 14, 2023
  
  be1e2163
- Fix bugs in scheduler · 7e5f604e
  Woosuk Kwon authored Feb 14, 2023
  
  7e5f604e
- Fix a bug in swap_in and swap_out · 1f739f9b
  Woosuk Kwon authored Feb 14, 2023
  
  1f739f9b
- Fix can_swap_in · b1644f76
  Woosuk Kwon authored Feb 14, 2023
  
  b1644f76
13 Feb, 2023 4 commits
- Fix typo · 531e1c74
  Woosuk Kwon authored Feb 13, 2023
  
  531e1c74
- Fix scheduler · d3d31766
  Woosuk Kwon authored Feb 13, 2023
  
  d3d31766
- Add scheduler · 5a309bb5
  Woosuk Kwon authored Feb 13, 2023
  
  5a309bb5
- Fix can_swap_in · eb52db1b
  Woosuk Kwon authored Feb 13, 2023
  
  eb52db1b