Commits · c3442c1f6fabe54adb82d2d676920c5f31b9834e · norm / vllm

20 May, 2023 1 commit
- Refactor system architecture (#109) · c3442c1f
  Woosuk Kwon authored May 20, 2023
  
  c3442c1f
19 May, 2023 1 commit
- Use runtime profiling to replace manual memory analyzers (#81) · f756799b
  Zhuohan Li authored May 19, 2023
  
  f756799b
15 May, 2023 1 commit
- Add docstrings to some modules and classes (#100) · b322fd16
  Woosuk Kwon authored May 14, 2023
  
  b322fd16
11 May, 2023 1 commit
- Implement presence and frequency penalties (#95) · 55f8b0a5
  Woosuk Kwon authored May 10, 2023
  
  55f8b0a5
10 May, 2023 2 commits
- Log system stats (#90) · e3319577
  Woosuk Kwon authored May 10, 2023
  
  e3319577
- Rename variables and methods (#91) · 8d66a7b6
  Woosuk Kwon authored May 10, 2023
  
  8d66a7b6
09 May, 2023 1 commit
- Refactor system architecture (#82) · 7c041ab5
  Woosuk Kwon authored May 09, 2023
  
  7c041ab5
05 May, 2023 1 commit
- Replace FlashAttention with xformers (#70) · c9d5b6d4
  Woosuk Kwon authored May 05, 2023
  
  c9d5b6d4
03 May, 2023 1 commit
- New weight loader without np copy (#52) · 27f1410d
  Zhuohan Li authored May 03, 2023
  
  27f1410d
30 Apr, 2023 1 commit
- Add an option to launch cacheflow without ray (#51) · 4858f3bb
  Zhuohan Li authored Apr 30, 2023
  
  4858f3bb
09 Apr, 2023 1 commit
- Add an option to use dummy model weights (#33) · ee88a7e5
  Woosuk Kwon authored Apr 08, 2023
  
  ee88a7e5
08 Apr, 2023 1 commit
- Implement block copy kernel to optimize beam search (#32) · 0f40557a
  Woosuk Kwon authored Apr 07, 2023
  
  0f40557a
05 Apr, 2023 1 commit
- Add CUDA graph-based all reduce launcher (#26) · 12659a0b
  Woosuk Kwon authored Apr 05, 2023
  
  12659a0b
02 Apr, 2023 1 commit
- Optimize data movement (#20) · 897cb2ae
  Woosuk Kwon authored Apr 02, 2023
  
  897cb2ae
21 Mar, 2023 1 commit
- Support tensor parallel (#2) · 2f49f155
  Zhuohan Li authored Mar 22, 2023
  
  2f49f155
13 Mar, 2023 1 commit
- Add miscellaneous updates (#8) · cfae35b8
  Woosuk Kwon authored Mar 13, 2023
  
  cfae35b8
10 Mar, 2023 1 commit
- Support beam search & parallel generation (#7) · 1a7eb7da
  Woosuk Kwon authored Mar 10, 2023
  
  1a7eb7da
01 Mar, 2023 1 commit
- Implement `single_query_cached_kv_attention` kernel (#3) · 0deacbce
  Woosuk Kwon authored Mar 01, 2023
  
  0deacbce
23 Feb, 2023 5 commits
- Set default dtype to half · 1ce13335
  Woosuk Kwon authored Feb 23, 2023
  
  1ce13335
- Minor · fdd0f2f4
  Woosuk Kwon authored Feb 23, 2023
  
  fdd0f2f4
- Add controller · 1f6c7ef4
  Woosuk Kwon authored Feb 23, 2023
  
  1f6c7ef4
- Add seq_ids to input metadata · 343cea3d
  Woosuk Kwon authored Feb 23, 2023
  
  343cea3d
- Fix slot mapping · 4b1ac23f
  Woosuk Kwon authored Feb 23, 2023
  
  4b1ac23f
22 Feb, 2023 2 commits
- Add Worker class · 8290fce4
  Woosuk Kwon authored Feb 22, 2023
  
  8290fce4
- Move worker/models -> models · 709a6917
  Woosuk Kwon authored Feb 22, 2023
  
  709a6917
16 Feb, 2023 5 commits
- Implement cache ops · 6f058c7b
  Woosuk Kwon authored Feb 16, 2023
  
  6f058c7b
- Minor · a1c67e6d
  Woosuk Kwon authored Feb 16, 2023
  
  a1c67e6d
- Fix return type error · 9e68a682
  Woosuk Kwon authored Feb 16, 2023
  
  9e68a682
- Add warning · 8edcabc7
  Woosuk Kwon authored Feb 16, 2023
  
  8edcabc7
- Fix KVCache shape · 2f4887de
  Woosuk Kwon authored Feb 16, 2023
  
  2f4887de
13 Feb, 2023 3 commits
- Fix get_model · ee944251
  Woosuk Kwon authored Feb 13, 2023
  
  ee944251
- Add model_utils · fffa2e1f
  Woosuk Kwon authored Feb 13, 2023
  
  fffa2e1f
- Fix cache engine · bb59a3e7
  Woosuk Kwon authored Feb 13, 2023
  
  bb59a3e7
09 Feb, 2023 2 commits
- Add cache engine · e7bee2aa
  Woosuk Kwon authored Feb 09, 2023
  
  e7bee2aa
- Add OPT · 39161c98
  Woosuk Kwon authored Feb 09, 2023
  
  39161c98