Commits · 3e9f991d6acd7efd90f04f1f530b837a40c93442 · norm / vllm

02 Mar, 2023 1 commit
- Use FlashAttention for `multi_query_kv_attention` (#4) · 3e9f991d
  Woosuk Kwon authored Mar 01, 2023
  
  3e9f991d
01 Mar, 2023 1 commit
- Implement `single_query_cached_kv_attention` kernel (#3) · 0deacbce
  Woosuk Kwon authored Mar 01, 2023
  
  0deacbce
25 Feb, 2023 1 commit
- Fix a bug in tying OPT embeddings (#1) · cbf8779a
  Woosuk Kwon authored Feb 24, 2023
  
  cbf8779a
24 Feb, 2023 11 commits
- Add README · c84c708a
  Woosuk Kwon authored Feb 24, 2023
  
  c84c708a
- Clean up the server script · fa16389a
  Woosuk Kwon authored Feb 24, 2023
  
  fa16389a
- [Minor] Fix printing format · 6aef2278
  Woosuk Kwon authored Feb 24, 2023
  
  6aef2278
- Add Frontend · 1132fae0
  Woosuk Kwon authored Feb 24, 2023
  
  1132fae0
- Add max_num_steps to SamplingParams · 46ce1356
  Woosuk Kwon authored Feb 24, 2023
  
  46ce1356
- Add is_finished · b39f149a
  Woosuk Kwon authored Feb 24, 2023
  
  b39f149a
- Merge pre_step and step · ef6098ec
  Woosuk Kwon authored Feb 24, 2023
  
  ef6098ec
- Reduce the number of states in scheduler · 53f70e73
  Woosuk Kwon authored Feb 24, 2023
  
  53f70e73
- Refactor and annotate types for attention · 762fd1c3
  Woosuk Kwon authored Feb 24, 2023
  
  762fd1c3
- Remove xformers · 7f22f90e
  Woosuk Kwon authored Feb 24, 2023
  
  7f22f90e
- [WIP] Add server script · afdbe5d3
  Woosuk Kwon authored Feb 24, 2023
  
  afdbe5d3
23 Feb, 2023 19 commits
- Fix attention · 932844f1
  Woosuk Kwon authored Feb 23, 2023
  
  932844f1
- Fix attention · ba84b872
  Woosuk Kwon authored Feb 23, 2023
  
  ba84b872
- Fix attention · 87e0bcd4
  Woosuk Kwon authored Feb 23, 2023
  
  87e0bcd4
- Set default dtype to half · 1ce13335
  Woosuk Kwon authored Feb 23, 2023
  
  1ce13335
- Fix sampler · de0fabbc
  Woosuk Kwon authored Feb 23, 2023
  
  de0fabbc
- Minor · fdd0f2f4
  Woosuk Kwon authored Feb 23, 2023
  
  fdd0f2f4
- Consider pempty tensor · 7f985166
  Woosuk Kwon authored Feb 23, 2023
  
  7f985166
- Fix typo · 86f9eb6d
  Woosuk Kwon authored Feb 23, 2023
  
  86f9eb6d
- Add controller · 1f6c7ef4
  Woosuk Kwon authored Feb 23, 2023
  
  1f6c7ef4
- Add unoptimized OPT Attention · d4bc1a4d
  Woosuk Kwon authored Feb 23, 2023
  
  d4bc1a4d
- Add greedy sampler · b56b6ca0
  Woosuk Kwon authored Feb 23, 2023
  
  b56b6ca0
- Add seq_ids to input metadata · 343cea3d
  Woosuk Kwon authored Feb 23, 2023
  
  343cea3d
- Add get_block_table · 4f6f4967
  Woosuk Kwon authored Feb 23, 2023
  
  4f6f4967
- Implement scheduler.step & Add a threshold for batch size · 331fa0b0
  Woosuk Kwon authored Feb 23, 2023
  
  331fa0b0
- decoding.py -> sampling_params.py · 501c4bd0
  Woosuk Kwon authored Feb 23, 2023
  
  501c4bd0
- DecodingParams -> SamplingParams · 86c682cd
  Woosuk Kwon authored Feb 23, 2023
  
  86c682cd
- Add get_len · af16c050
  Woosuk Kwon authored Feb 23, 2023
  
  af16c050
- Move max_context_len · d0945122
  Woosuk Kwon authored Feb 23, 2023
  
  d0945122
- Fix slot mapping · 4b1ac23f
  Woosuk Kwon authored Feb 23, 2023
  
  4b1ac23f
22 Feb, 2023 4 commits
- Add Worker class · 8290fce4
  Woosuk Kwon authored Feb 22, 2023
  
  8290fce4
- Add input metadata · 7b6844e5
  Woosuk Kwon authored Feb 22, 2023
  
  7b6844e5
- Minor · 608f74ff
  Woosuk Kwon authored Feb 22, 2023
  
  608f74ff
- Move worker/models -> models · 709a6917
  Woosuk Kwon authored Feb 22, 2023
  
  709a6917
18 Feb, 2023 2 commits
- Add tests for kernels · af68ec1c
  Woosuk Kwon authored Feb 18, 2023
  
  af68ec1c
- Add reshape_and_cache op · c413c41c
  Woosuk Kwon authored Feb 18, 2023
  
  c413c41c
16 Feb, 2023 1 commit
- cache_kernel -> cache_kernels · ffad4e1e
  Woosuk Kwon authored Feb 16, 2023
  
  ffad4e1e