Commits · a1b3de86cd6f27aeb299d45296a7409b8d2b7c0c · kecinstone / 2024pra-vllm

"cacheflow/engine/llm_engine.py" did not exist on "aedba6d5ec4e1eaad10745f71970c10b601f9dc1"

29 Mar, 2023 1 commit
- FastAPI-based working frontend (#10) · 721fa3df
  Zhuohan Li authored Mar 29, 2023
  
  721fa3df
26 Mar, 2023 1 commit
- Minor · d359cda5
  Woosuk Kwon authored Mar 26, 2023
  
  d359cda5
21 Mar, 2023 1 commit
- Support tensor parallel (#2) · 2f49f155
  Zhuohan Li authored Mar 22, 2023
  
  2f49f155
13 Mar, 2023 1 commit
- Add miscellaneous updates (#8) · cfae35b8
  Woosuk Kwon authored Mar 13, 2023
  
  cfae35b8
12 Mar, 2023 1 commit
- Add memory analyzer & utomatically configure KV cache size (#6) · e9d3f2ff
  Woosuk Kwon authored Mar 11, 2023
  
  e9d3f2ff
10 Mar, 2023 1 commit
- Support beam search & parallel generation (#7) · 1a7eb7da
  Woosuk Kwon authored Mar 10, 2023
  
  1a7eb7da
06 Mar, 2023 1 commit
- Fix a bug in 1D input shape (#5) · 04e5acc0
  Woosuk Kwon authored Mar 06, 2023
  
  04e5acc0
02 Mar, 2023 1 commit
- Use FlashAttention for `multi_query_kv_attention` (#4) · 3e9f991d
  Woosuk Kwon authored Mar 01, 2023
  
  3e9f991d
01 Mar, 2023 1 commit
- Implement `single_query_cached_kv_attention` kernel (#3) · 0deacbce
  Woosuk Kwon authored Mar 01, 2023
  
  0deacbce
25 Feb, 2023 1 commit
- Fix a bug in tying OPT embeddings (#1) · cbf8779a
  Woosuk Kwon authored Feb 24, 2023
  
  cbf8779a
24 Feb, 2023 2 commits
- Refactor and annotate types for attention · 762fd1c3
  Woosuk Kwon authored Feb 24, 2023
  
  762fd1c3
- Remove xformers · 7f22f90e
  Woosuk Kwon authored Feb 24, 2023
  
  7f22f90e
23 Feb, 2023 12 commits
- Fix attention · 932844f1
  Woosuk Kwon authored Feb 23, 2023
  
  932844f1
- Fix attention · ba84b872
  Woosuk Kwon authored Feb 23, 2023
  
  ba84b872
- Fix attention · 87e0bcd4
  Woosuk Kwon authored Feb 23, 2023
  
  87e0bcd4
- Set default dtype to half · 1ce13335
  Woosuk Kwon authored Feb 23, 2023
  
  1ce13335
- Fix sampler · de0fabbc
  Woosuk Kwon authored Feb 23, 2023
  
  de0fabbc
- Minor · fdd0f2f4
  Woosuk Kwon authored Feb 23, 2023
  
  fdd0f2f4
- Consider pempty tensor · 7f985166
  Woosuk Kwon authored Feb 23, 2023
  
  7f985166
- Fix typo · 86f9eb6d
  Woosuk Kwon authored Feb 23, 2023
  
  86f9eb6d
- Add unoptimized OPT Attention · d4bc1a4d
  Woosuk Kwon authored Feb 23, 2023
  
  d4bc1a4d
- Add greedy sampler · b56b6ca0
  Woosuk Kwon authored Feb 23, 2023
  
  b56b6ca0
- Add seq_ids to input metadata · 343cea3d
  Woosuk Kwon authored Feb 23, 2023
  
  343cea3d
- Fix slot mapping · 4b1ac23f
  Woosuk Kwon authored Feb 23, 2023
  
  4b1ac23f
22 Feb, 2023 3 commits
- Add input metadata · 7b6844e5
  Woosuk Kwon authored Feb 22, 2023
  
  7b6844e5
- Minor · 608f74ff
  Woosuk Kwon authored Feb 22, 2023
  
  608f74ff
- Move worker/models -> models · 709a6917
  Woosuk Kwon authored Feb 22, 2023
  
  709a6917