Commits · 84eee24e20ff4c0fc1b126289265f560089efa47 · norm / vllm

12 Apr, 2023 1 commit
- Collect system stats in scheduler & Add scripts for experiments (#30) · 84eee24e
  Woosuk Kwon authored Apr 12, 2023
  
  84eee24e
10 Apr, 2023 1 commit
- Support block size 32 (#35) · b9926f7f
  Woosuk Kwon authored Apr 09, 2023
  
  b9926f7f
09 Apr, 2023 1 commit
- Add an option to use dummy model weights (#33) · ee88a7e5
  Woosuk Kwon authored Apr 08, 2023
  
  ee88a7e5
08 Apr, 2023 1 commit
- Implement block copy kernel to optimize beam search (#32) · 0f40557a
  Woosuk Kwon authored Apr 07, 2023
  
  0f40557a
06 Apr, 2023 1 commit
- Fix potential bugs in FastAPI frontend and add comments (#28) · a490aafa
  Zhuohan Li authored Apr 06, 2023
  
  a490aafa
05 Apr, 2023 1 commit
- Add CUDA graph-based all reduce launcher (#26) · 12659a0b
  Woosuk Kwon authored Apr 05, 2023
  
  12659a0b
02 Apr, 2023 2 commits
- Optimize data movement (#20) · 897cb2ae
  Woosuk Kwon authored Apr 02, 2023
  
  897cb2ae
- Merge QKV into one linear layer (#15) · 1f01a18d
  Zhuohan Li authored Apr 02, 2023
  
  1f01a18d
01 Apr, 2023 1 commit
- Use FP32 for log probabilities (#19) · a90c97d7
  Woosuk Kwon authored Mar 31, 2023
  
  a90c97d7
31 Mar, 2023 2 commits
- Add custom kernel for RMS normalization (#16) · 09e92454
  Woosuk Kwon authored Mar 31, 2023
  
  09e92454
- Optimize tensor parallel execution speed (#17) · c45f3c3a
  Zhuohan Li authored Apr 01, 2023
  
  c45f3c3a
30 Mar, 2023 3 commits
- Implement preemption via recomputation & Refactor scheduling logic (#12) · 7a7929ab
  Woosuk Kwon authored Mar 30, 2023
  
  7a7929ab
- Implement custom kernel for LLaMA rotary embedding (#14) · 88c0268a
  Woosuk Kwon authored Mar 30, 2023
  
  88c0268a
- Implement LLaMA (#9) · 80a2f812
  Woosuk Kwon authored Mar 29, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  80a2f812
29 Mar, 2023 2 commits
- Add cache watermark to avoid frequent cache eviction (#11) · 64e0e383
  Woosuk Kwon authored Mar 29, 2023
  
  64e0e383
- FastAPI-based working frontend (#10) · 721fa3df
  Zhuohan Li authored Mar 29, 2023
  
  721fa3df
26 Mar, 2023 1 commit
- Minor · d359cda5
  Woosuk Kwon authored Mar 26, 2023
  
  d359cda5
21 Mar, 2023 1 commit
- Support tensor parallel (#2) · 2f49f155
  Zhuohan Li authored Mar 22, 2023
  
  2f49f155
13 Mar, 2023 1 commit
- Add miscellaneous updates (#8) · cfae35b8
  Woosuk Kwon authored Mar 13, 2023
  
  cfae35b8
12 Mar, 2023 1 commit
- Add memory analyzer & utomatically configure KV cache size (#6) · e9d3f2ff
  Woosuk Kwon authored Mar 11, 2023
  
  e9d3f2ff
10 Mar, 2023 1 commit
- Support beam search & parallel generation (#7) · 1a7eb7da
  Woosuk Kwon authored Mar 10, 2023
  
  1a7eb7da
06 Mar, 2023 1 commit
- Fix a bug in 1D input shape (#5) · 04e5acc0
  Woosuk Kwon authored Mar 06, 2023
  
  04e5acc0
02 Mar, 2023 1 commit
- Use FlashAttention for `multi_query_kv_attention` (#4) · 3e9f991d
  Woosuk Kwon authored Mar 01, 2023
  
  3e9f991d
01 Mar, 2023 1 commit
- Implement `single_query_cached_kv_attention` kernel (#3) · 0deacbce
  Woosuk Kwon authored Mar 01, 2023
  
  0deacbce
25 Feb, 2023 1 commit
- Fix a bug in tying OPT embeddings (#1) · cbf8779a
  Woosuk Kwon authored Feb 24, 2023
  
  cbf8779a
24 Feb, 2023 9 commits
- [Minor] Fix printing format · 6aef2278
  Woosuk Kwon authored Feb 24, 2023
  
  6aef2278
- Add Frontend · 1132fae0
  Woosuk Kwon authored Feb 24, 2023
  
  1132fae0
- Add max_num_steps to SamplingParams · 46ce1356
  Woosuk Kwon authored Feb 24, 2023
  
  46ce1356
- Add is_finished · b39f149a
  Woosuk Kwon authored Feb 24, 2023
  
  b39f149a
- Merge pre_step and step · ef6098ec
  Woosuk Kwon authored Feb 24, 2023
  
  ef6098ec
- Reduce the number of states in scheduler · 53f70e73
  Woosuk Kwon authored Feb 24, 2023
  
  53f70e73
- Refactor and annotate types for attention · 762fd1c3
  Woosuk Kwon authored Feb 24, 2023
  
  762fd1c3
- Remove xformers · 7f22f90e
  Woosuk Kwon authored Feb 24, 2023
  
  7f22f90e
- [WIP] Add server script · afdbe5d3
  Woosuk Kwon authored Feb 24, 2023
  
  afdbe5d3
23 Feb, 2023 6 commits
- Fix attention · 932844f1
  Woosuk Kwon authored Feb 23, 2023
  
  932844f1
- Fix attention · ba84b872
  Woosuk Kwon authored Feb 23, 2023
  
  ba84b872
- Fix attention · 87e0bcd4
  Woosuk Kwon authored Feb 23, 2023
  
  87e0bcd4
- Set default dtype to half · 1ce13335
  Woosuk Kwon authored Feb 23, 2023
  
  1ce13335
- Fix sampler · de0fabbc
  Woosuk Kwon authored Feb 23, 2023
  
  de0fabbc
- Minor · fdd0f2f4
  Woosuk Kwon authored Feb 23, 2023
  
  fdd0f2f4